Re: The wirehead problem

Tim Tyler <seemysig@xxxxxxxxxxxxxx> wrote:
Curt Welch wrote:
Tim Tyler <seemysig@xxxxxxxxxxxxxx> wrote:
Curt Welch wrote:

But if they are a real risk to our survival, and we understand that
risk, then we simply won't use them. If using the technology
actually produced a net reduction to our survival (it's danger was
worse than it's advantage to us) we simply won't use them.
Worse - on average? If so, consider cigarettes. Cigarettes benefit
the few at the expense of the many. It seems as though your argument
would suggest that humans simply don't use cigarettes.

Cigarettes are an example of the wirehead problem. We are not wired
directly for "survival" because there is no simple way to wire a
reinforcement learning machine for survival. As such, evolution has
wired us with a set of needs that correlate very well to the survival
of our genes - such as skin and body damage sensors, and low energy
level sensors. WE don't act to survive. We act to prevent the things we
are wired for - to prevent ruining out of food, or having our body
damaged, etc. Cigarettes are something that showed up in the
environment only recently and represent a disconnect between our hard
wired motivations and evolution's higher goal of survival. Evolution
will change us in time to stop our cigarette use if our intelligence
doesn't adjust that behavior for us. Its hard for our intelligence to
get it right because of the very long time delay between cigarette use
and the pain it causes (lung cancer or stroke).

It seems as though you acknowledge that, in practice, humans sometimes
do use technology - despite a net negative effect on their survival.

Only when they fail to understand the true danger of a technology
(asbestos) or when the technology allows them to wirehead themselves
(cigarettes).

What happens is that those who benefit (manufacturers, taxation
bodies) are successful in creating and marketing the technology.

I think the ability of one person to "trick" another like that is a trivial
short term effect. Those tricks for the most part only work, then making
the wrong choice is mostly harmless - like which brand of soap we use.

Though I don't deny that the power of marketing shapes the direction a
society takes, I would argue that no matter how good the marketing is, it
can't force society down a path it wasn't already pre-disposed to follow.
No amount of marketing will make eating dog *** popular. No amount of
marketing will make jumping off buildings popular.

Strong marketing can make some people eat dog ***, or jump off a building,
but it can't force the majority of a society to do things they weren't
already pre-disposed to do.

The next question is: could something like that happen for synthetic
intelligences?

Only if the true danger was hidden from people. Such as if the AIs were
good until they got to the point they were in a clear majority, and then
suddenly turned bad at a rate which was too quick for the humans to stop.
And clearly, a company selling robots would want to try and cover up any
indications that the robots would sometimes turn bad and continue to try
and use every marketing trick in the book to get people to buy more of
them.

But it would not be marketing which was the main force at work there. It
was the fact that the _long_term_ danger of robots weren't obvious to
people until it was too late (like with asbestos). The motivations of the
company just kept the truth hidden a little longer. In the end the truth
would come out - and we can only hope that if a such a surprise is waiting
for us with the AI robots, it will come out before it's too late to fix.

We can't answer for sure how we will end up using AI technology until
we have the technology, and can estimate it's true value to us. My
best guess is the type of AI technology you seem to think we will end
up (something that will out-evolove us in short order) is not
something we will love, but something man will greatly fear exactly
because it has the power to out-evolove us in short order [...]
Probably true for some people. The luddites will always hate
technology, no matter how great it is. However, I don't see
this as being true in general. People will probably mostly
look back on the era of tooth decay and broken bones and be
thankful for their new circumstances.

It's not a fear of technology that is the issue here. It's the fear of
a technology that does far greater harm than good - this is a fear that
is true of everyone already - not of just the "luddites". It's why we
put blade guards on power saws for example. [...]

There are many tools which are so dangerous that you can't even buy
them (like nuclear rectors for the home). The fact that these don't
exist has nothing to do with a Luddite fear of technology. It's the
simple fact that when a technology does more potential harm than good,
we simply don't build it - because no one will buy it.

Well, I think the superintelligence will be popular. It will /have/ to
be popular to start with, for its customers to pay it any money. Think
of something like Google on steroids. Google is a popular company - it
helps many people - and, well, at least it's not evil.

Or it at least tries not to be too evil. Many people on Usenet think they
are doing serious harm to Usenet by not effectively blocking spammers.

So, I'm saying in my scenarios, usually, people love the
superintelligence. It cleans up the environment, provides endless cheap
power, irrigates the planet, and generally saves the world.

You seem to be saying that - in my scenarios, people won't like the
superintelligence. I don't see that. A military superintelligence
might not be so popular - but that is not the type I am mostly
concerned with.

No, that's not what I'm saying at all.

I'm saying the AI we do build, will be loved, and will do all the things
you say it will. But it will do under 100% control by humans. We don't
pay _it_ anything - just like we don't pay our chair for holding our ***
off the ground. And these AIs, the ones we have 100% control over, will
never replace humans, they will remain as our tools.

I'm saying the AI that you want to build - the one that will take over the
world - we will never be build, and if anyone did build that type of AI, we
would quickly learn to fear it because we would see many little examples of
how easily that type of AI can turn bad and kill humans or otherwise harm
the things that humans love.

Humans are the ones pushing to build these machines. Basically
the machines offer things of value to those that help construct
them - and to those that use them - in the form of wealth, power,
control etc.

At every point, there are human genes benefitting from their
association with the machines.

If you can't control the machine, it won't offer anyone wealth or
power.

If you can't control it at all, indeed, it probably won't.

It will make itself rich by taking resources away from the humans. And
when it figures out that only the machines are the ones with all the
wealth and power, it will stop making food for humans because there's
no money in that - the real money is in the battery and home nuclear
power business which all the wealthy robots are buying.

So, what's with the premise? Humans will build these machines,
so why would they not be able to get them to do what they want?

The problem is not with machines being out of control. It's
more to do with the fact that humans compete with other humans,
and want to rule the world - and the best way to do that is at
the head of a huge robot army.

Well, that is a problem. Humans will use the technology to create great
wealth and power for themselves. And because you don't need humans to do
your fighting or work for you, these people that get themselves into a
position of controlling a huge AI work-force and army, could easily just
decide to take over the world and kill most the humans because they might
only want say 100,000 of their "close" friends around and the rest of the
humans on the planet can just just die (like Hitler tried to do).

But if the AIs are built to be total slaves and don't have a will to
reproduce or survive on their own, they only have a will to server this new
Hitler, then the AIs just won't take over. At some point the humans might
kill most of each other leaving a gene pool too small to survive and then
they just die out. But the machines still won't take over because they have
no motivation to do so. They will just turn themselves off and sit in the
corner and do nothing - like our TVs will do.

Your idea is that the AI machines we build and love will have a natural
human-like will to survive. I don't believe we will build that type of
machine because it is simply to dangerous to build and because we don't
need to build that type of machine to get the full advantage of having
intelligent machines solving our problems for us.

Some tiny fraction of all modern human genes might even wind up
perpetuated in museums across the galaxy - in ancestor simulations.
Surely that would represent an incredible triumph of human gene
success - in terms of sheer numbers - by modern standards.

:)

I can see that playing out in a million years, but not in 100 or 1000.

It takes at least 70 thousand years to conquer the galaxy - according
to my sums.

If you build intelligent
machines, and allow them a place in our society (voting rights in a
government for example - or property ownership rights), they will
simply take everything you own away from you in time, including your
own life.
Unless they are built to value that, I would presume.

I don't believe it's possible to build abstract values into intelligent
machines. [...]

They seemed to manage it with Deep Blue. Or do you mean to say that
Deep Blue was not "intelligent" in your sense of the word - in which
case, what /do/ you mean?

I don't consider the value of "follow the game graph of chess to a win
state" to be an "abstract value". At least not abstract enough to be what
I was talking about.

Like in the other post where we are talking about suicide. How do we
hard-code the value of "don't commit suicide" into a machine? That is
clearly very difficult compared to trivial to specify value of "win the
chess game".

Humans have a very strong will to survive, and if something can
> wipe us out by taking control of the earth away from us, they
> must have a much stronger will to survive than humans do.

I don't know about "will to survive" - but machines will have
better technology, better designs, and less of a hangover - which
will pretty-much explains why they will prevail.

Yes, if given the chance, they could easily take over and prevail. I
don't think humans as a whole will allow it however so we will stay far
far away from that very dangerous and slippery slope. We will put up
guard rails all over the place to make sure someone is not creating a
society of dangerous AI robots.

Success in prohibiting an AI society would most likely represent an
epic disaster for our civilization. The preservation of man would
likely hold back evolution, cripple our spaceworthyness, and increase
the risk of our civilization's extinction. I hope - and expect -
that we will not be that stupid.

Well, again, this requires us to answer the question: "Are we a society of
intelligent agents, or our we a society of humans"?

Your view seems fairly clearly to put us as a society of intelligent agents
where our views and ideas (memes) are the only thing which is important and
not our bodies. I don't think humans as a whole will ever agree with you.
But I could be wrong.

I think the prime view of humans is "protect your body", unlike the prime
view you seem to have which is "protect your ideas".

As such, if "protect your body" is the true prime motivation, then we won't
allow AI robots to take over our society. But if "protect your ideas" is
the goal, then allowing a more advanced race of AI robots to carry the
ideas (our society of ideas) into the future for us is a clear win for the
ideas.

People are highly protective of their ideas, but only because the ideas are
their prime tools for protecting their bodies. Any idea which doesn't
protect their bodies will be eliminated. And the idea of allowing super
intelligent AIs to take control of the earth is an idea what is
incompatable with protecting our bodies.

If you build robots which have the basic need of not letting itself be
damaged, and a basic need to reproduce, and we make it far more
intelligent than any human, humans will be screwed in no time flat.

Right. Now substitute "robot" for "meme", and identify any differences
you see. The new replicators are here *already*. Their phenotypes are
here *already*. Self-replicating artificial agents are not something
we will need to avoid in the future - they are here, now, and have been
here for thousands of years. They are directly responsible for all
our recent progress. They are swarming in exponentially-increasing
numbers. They are responsible for an ever-increasing share of the
economy. They created our big brains, to replicate themselves in,
our computer networks to replicate themselves more effectively in,
and soon they will have superintelligence - at which point the
construction scaffolding will be ready to be kicked away, and our
efforts on their behalf will be done.

Memes get their power and their _motivation_ from the hardware they exist
in. Your view requires that the meme is independent of the hardware which
it can never be. Memes can only exist if the hardware lets them exist.
And the motivation of the hardware is not for the survival of the meme, but
instead, for the survival of the hardware itself. Any meme which works
against the survival of the hardware will be shunned and rejected - just
like computer virus are shunned and rejected by our computer hardware.

If the memes were in control, the computer virus memes would have taken
over our computers and made them useless by now. But they have failed to
do that in a big away because they are not in control.

I follow your logic to the end of your sentence - and predict the
rapid demise of humans as a consequence - but you don't seem to.

As far as I can tell, the logic is that DNA has some kind of
priviledged status.

Yes. It's privileged simply because it's currently in control and "kings"
don't just step down and walk away the minute a better king comes along
which is what you seem to be suggesting will happen. It will fight to hold
onto it's control for as long as it can. You don't open your gates and
give weapons to the other king when you don't have to. You kill him before
he gets powerful enough to take the power away from you. DNA is the
current king, and it's going to try and kill any AI robot that comes along
and tries to take power away from it. It certainly won't be stupid enough
to hand the keys of the castle over to the AI robots.

My perspective is that replicators have lost out to better
replicators before - see RNA - and that there's *absolutely nothing*
stopping it from happening again.

Well, again, in the long run, I can easily see DNA loosing out to other
replicating _hardware_ like AI robots simply because DNA isn't as good as
intelligent design. But the long run is more like thousands of years, not
the 100 year time frame you talk about. I don't believe DNA will "give up"
without a fight (so to say). It's got a clear lead at the moment and there
is no short term reason for it (humans) to give up that lead. It's not
like we are being invaded by intelligent alien robots that are here to take
the resources of earth away from us. We are talking about intentionally
building them to create our own destruction. Humans are smart enough to
not do that.

I think humans are built to value the protection of the human body over the
protection of human society or human ideas (memes). Society breaks down
quickly when people's lives are in danger (like in times of a natural
disaster). It's because given the choice to preserve some of societies
ides over their own body, people will normally choose protecting their own
skin (or the skin of their offspring) over all else.

If we can't remain in 100% control of the intelligent machines, we won't
build them. We will only build the machines that we can remain in control
of. That's how I see it.

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.