Re: What are we going to wish for?

Curt Welch wrote:
Tim Tyler <seemysig@xxxxxxxxxxxxxx> wrote:
casey wrote:
On Apr 1, 8:22 pm, Tim Tyler <seemy...@xxxxxxxxxxxxxx> wrote:

IMO, that is not what's going to happen. Instead, we
will see AIs built to maximise individual corporate
or governmental goals - not those of individual humans,
or of the whole of humanity.

Corporations are already AIs. They are functioning reinforcement learning
systems motivated to maximize reward as measured by the flow of the
abstract measure of reward we built them to maximize - money. In a
functioning capitalistic society they make themselves happy by making their
customers happy, and in the end, this makes all the individual humans
happy. We are all linked together through the large external reward system
of money.

But these giant and very smart and very powerful AIs only work for us as
long as we keep control of them - as long as we keep control of their
rewards. If we loose control of them, we become slaves to them, instead
of them being slaves to us. Some people think that since humans are the
prime engine of the corporations (currently - won't be true after we invent
strong AI), that the corporations won't ultimately take over. But that's
wrong. Humans are easily manipulated by our reward systems and the
corporation is more than willing to force it's employees to do whatever the
corporation needs to have done - including making the CEO care more about
the needs of the corporation than about the needs of other humans by paying
him so much he can't resist doing anything else.

The idea of "corporations winning" seems to pretty clearly
suggest the possibility of humans not doing so well - since
most corporations will probably wind up being largely
descended from today's machines.

If corporate goals, the monopolies and mergers
commision may find themselves with some work to do.

If governmental goals, the rest of the world had
better watch out.

Well, advancing technology will both allow, and force, the world to merge
together under larger systems of government until we reach the point of a
true world government. The development of AI will certainly help allow and
force that to happen. At which time, the AIs will become slaves to the
needs of the government, which will be controlled by the humans of the
world.

That seems like rather a detailed prediction. We will probably
get a world government at some stage, but who is going to be
in charge of it - and whether it will be controlled by
"the humans of the world" - seem pretty speculative to me.

Why not something more like the current situation - with
a huge monopolistic governmental infrastructure in control -
and a few symbolic pieces shuffled around now and again
to give the appearance of a democracy?

As I have suggested before, but I don't think you fully grasped, (because
you accused me of wanting to get rid of capitalism when that's not my
intent at all), I believe we will merge our "voting" and "money spending"
into a single system of control where we all get an equal vote in the form
a fixed amount of money per person to spend (aka vote) on what we want the
government (run by AIs) to do for us.

Capitalism is economic natural selection. If that continues, we
will have survival of the fittest among AIs - and unmodified
humans will rapidly become war casualties - since huge, powerful
machine-like organisms represent a _huge_ attractor to any
evolutionary process - and even a cursory glance at the
world shows the planet in the process of getting sucked in.

The only way I see of avoiding that is to ditch capitalism -
and to put an end to natural selection.

Not that I think avoiding it is desirable - rather the opposite:
attempts to avoid such a scenario may have the potential to
permanently cripple our future evolutionary potential.

The current form of capitalism only works as long as humans are a key part
of the system doing all the production of goods and services. But as AIs
get to the point of being as intelligent as any human, we will find
ourselves in a situation where we are no longer a key part of the system.
This will create growing amounts of inequity between humans because our
wealth and power under the current system won't be a function of what we do
but will be a function of what we own and control. Our time and labor will
be worth less and less as technology advances creating an every growing
divide between the super rich and the super poor. Hard work will have less
and less to do with getting the "good life". Who you know, and what you
own (or what your family owns) will be the only key to happiness. The
families and groups that own all the AIs will end up controlling the world
and unlike now, they will have no need for laborers to build their big
homes and keep them fed and take care of them because they will have AIs to
do all that for them.

But I don't think the world is going to let that happen, and the result is
that our governments will supply increasing higher levels of service to us
all and less will be done by the capitalistic systems directly. But this
will create inefficiencies which will be resolved by a transformation of
government services into a capitalistic system where people get to "vote"
on what services they get with some form of "money" (call them government
vouchers if you like?). So the production system is still regulated and
optimized by the flow of rewards (capitalism), but no humans aren't doing
the production - they are just telling the machine (aka the government)
what they want done.

History suggests growing inequalities - but also, increases
in the minimum standard of living. A fairly conservative
projection would be to simply extend those trends into the
future. However, the sheer scale of the coming changes
makes such projections unreliable.

Your map of the future seems rather specific to me. It
doesn't make sense to give a lot of details, since many of
them represent current unknowns - and your account fails
to indicate the difference between things that are likely
and things that are uncertain - it is all written like
future history :-(

In either case, we had best hope the organisation
in question knows how to properly phrase wishes -
so that the AI doesn't too carelessly slurp up
the solar system and turn it into utility - while
its creators are screaming "No! that's not what
I /meant/ to ask for!"

:)

We prevent that from happening by never letting go of the flow of rewards.
We don't have to "put our order in" now, and never ask again. We ask and
vote on what we want every time we give them a reward - ever time we "pay"
them to do something for us.

Easier said than done, when the AI is many times smarter than you
are.

And one of the things we will always spend a lot of our "money" on, is to
make sure we never lose control of their rewards. We will monitor what
happens, and we will ask the AIs to build AIs to monitor other AIs and
report to us. We will build AI police forces which police the activity of
the other AIs (if that is needed). Whatever is needed to keep a leash on
the AIs, we will do.

Building bigger AIs to control our AIs is not much of a solution - who
controls the bigger AIs? It would be like putting the cocaine in a vault
guarded by heroin addicts. OK - but where is the heroin kept?

Long before humanity loses control of the whole system, we will see what
happens when we lose control of an individual AI (and it goes off killing
humans for example). These will be the test cases that will let us know
what to watch out for in the long run.

It would be nice if we had multiple shots at the problem. But it
is far from clear that we do. The first seed AI may successfully
slurp up the compute cloud. Other similar projects may then
be starved of resources - and face severe established competition.

The AI could be on a server farm - and need not be open source.
It could be deployed in robots using tamper-proof hardware.
The first seed AI may simply win - by getting rapidly into a
position where it is far enough ahead to choke any prospective
competitors. IT is often naturally a winner-take all ecosystem -
and AI may be like that only more so.
--
__________
|im |yler http://timtyler.org/ tim@xxxxxxxxxxx Remove lock to reply.
.

Relevant Pages

  • Re: What are we going to wish for?
    ... Corporations are already AIs. ... customers happy, and in the end, this makes all the individual humans ... long as we keep control of them - as long as we keep control of their ... of the government with our equal votes. ...
    (comp.ai.philosophy)
  • Re: Merging with machines inevitable, scientists say
    ... I see the odds of the machines taking over being exceptionally slim. ... Humans are programmed by evolution with the goal of surviving. ... build AIs with the goal of surviving. ... advanced AIs for the sole purpose of taking over control from the humans. ...
    (comp.ai.philosophy)
  • Re: Nuclear Power - the only answer.
    ... > That would make good sense if it were not for the fact that humans have not ... > yet even managed to control their intake of alcohol and as proof of that ... > more of them die from that cause than do from nuclear power generation. ... I think you look pretty trusting of your government which is nicer than ...
    (soc.culture.scottish)
  • Re: The wirehead problem
    ... Wipe out all the humans, ... machines will in short order die as well. ... in control right now, and our genes strongly motivate us not to give up ... There's no path I can see where humans will allow the AIs to gain the power ...
    (comp.ai.philosophy)
  • Re: Whos Surprised?
    ... That's the failure in your stance that NO government is the ... humans in a government vacuum. ... That's where we disagree. ... can live with, and support, backed by the force to control individual ...
    (rec.bicycles.racing)