Re: consciousness

JGCASEY <jgkjcasey@xxxxxxxxxxxx> wrote:

CW:
My rational logic tells me quite clearly that our feelings are
the behavior of a operant conditioned learning controller which
is evolving it's behavior for the purpose of maximising future
rewards. As such, anything that it can do, or sense which
creates higher future rewards, is reinforced and repeated.

JC:
Ok. So now we need to build a simple machine that is an example
of a operant conditioned learning controller that evolves its
behavior in such a way it increases its rewards.

Yes, I believe so. However, that alone is easy. I did that 25 years ago
(as have others even before I was born).

The hard part is making it evolve ever greater complexity of behavior
instead of fairly quickly hitting a limit of what it can learn. A classic
problem with more complex environments is that simple RL systems tend to
unlearn (aka forget) previous lessons when the environment changes. So it
can learn task A, then learn task B, but when it goes back to task A, all
the work learning task B has made it forget task A.

A very simple example would be an agent that has to learn to turn left to
get a reward. It learns it just fine and next thing you know, it's always
turning left. But then later, the environment changes, and now it must
turn right to get a reward. It first has to unlearn everything it knew
about turning left, then learn to turn right. Given time, it learns this
however. But when the environment changes and it's time to start turning
left, it's now been totally conditioned to only turn right. So it's got to
unlearn turning right before it can learn to turn left.

The problem here is that a simple machine isn't able to recognize these two
conditions as being a different context. It sees the "need to make a turn
now" condition as the same. So it tries to learn what is right. A better
machine will somehow see it as two different contexts, and learn that
turning right is the correct behavior for one context, and that turning
left is the correct behavior for the other context. A machine that can't
learn to recognize what context the environment is in can't solve these
problems.

How the learning machine comes to recognize the current context of the
environment, is a huge key to how well this type of machine can work. Most
people who have decided reinforcement learning was too simple, failed to
grasp that the they weren't using a sophisticated enough system for
recognizing context. Given a very complex and very high dimension
environment created by high dimension sensors, how do we make the machine
understand the correct context of the current environment? Because of the
high dimension sensors, the current sensory data will be unlike anything
seen in the past. You can't for example just take a snap shot of a video
frame and compare it byte for byte with future video frames to establish
that the current context is the same as some past context. Instead, the
system must create some type of measure of similarity or closeness to past
learning events. Is the current context more like those times where I
needed to turn left, or is it more like the times where I needed to turn
right? This is the hard problem that must be solved before an RL systems
will start producing interesting behavior.

TDgammon solved it by having the author of the program hand-code a sensory
environment mapping function that did a good job of solving the problem for
the game of Backgammon. But his technique can't be directly applied to
other environments. Every environment would have to have it's own mapping
function hand coded by the programmer.

But this can't be what our brain does, because the brain can produce good
mapping functions for environments never seen before (like Usenet). So, to
duplicate human level learning skills, we have to first duplicate human
level context mapping skills.

This is not a problem that behaviorists have shed much light on this for
us. How is the learning of one environment shared with other similar
environments? Behaviorists tell us that there is sharing, and that it can
be represented with a constant to show the degree of sharing. But I'm not
aware of any work that has made progress on predicting what the degree of
sharing would turn out to be.

In other words, what is the closeness function that the brain uses to
measure how close one environment is to all the past environments under
which learning has happened?

What's clear to me is that this closeness function is not hard coded. It's
learned as well. The brain is basically performing a type of compression
function on the sensory data and that closeness function is driven by this
compressed representation. The environment is basically broken down into
key parameters and the closeness of each parameter is measured. Finding
algorithms to do this at the level the brain is able to do it, is the key
to making reinforcement learning systems duplicate what the brain does. If
it doesn't have a very complex and rich classification of the environment
into attributes (or objects) it will have no hope of building complex and
rich behaviors that respond in reasonable ways to different environments.

What I've found, is that these distributed nets show great promise in how
this can be done. Each node of a network like mine represents a parameter
extracted from the sensory environment. The current outputs of every node
in the network can be looked at as a decoding of the the current sensory
environment into different parameters. Each node can then make a
"closeness" determination and produce behavior based on it's small view of
the context of the current environment. I think this approach answers how
the brain can "understand" such a huge set of contexts and do such a good
job of producing good behaviors in environments it's never seen before.
But the belief that the approach is workable, doesn't tell us the details
of what the nodes should be doing, and how they need to respond to
training.

My current nets already do this in theory - they just don't do a good
enough job to be interesting. But they demonstrate the power of the
approach, even if the current results are uninteresting otherwise.

CW:
They put down the ideas of behaviorism because to believe what
the behaviorists have been saying, is to accept the fact that we
are nothing more than machines that our shaped by our environment.

JC:
Ok. So now we need to build a simple machine that is shaped
by its environment.

Right, same thing. Do you not understand that reinforcement learning
machines are shaped by their environment?

[...]

JC:
You want others to see the pulses going left or right as
"making > decisions", "forming concepts" and so on but
we need to see its external behaviour as doing just that
in the same way we can see an animal doing just that.

CW:
Yeah, making it act like an animal even a little bit would
be a good club to work with. But I'm not there yet.

My frustration is not with how much I still have to do, my
frustration is that that there seems to be no one here that
can understand what I've done - and a huge part of why they
can't, is because like you, you are still under the influence
of these seductive idea drugs.

JC:
That I see as a cop out. Your notion that I cannot see your
nets doing things because I believe humans have "a special
power called subjective experience" is rubbish.

Could be. I'm just grasping at straws as to why you never seem to be able
to grasp the significance of the stuff I've written so many times (like
what I just wrote above about how the core problem here is to make RL
machines with strong systems to classify context and how my networks are a
huge step in the right direction of how to do that).

If your nets learn to play a good game of chess, or are able
to get a machine up and walking and then adapting if one of
its legs is ripped off I will be impressed. No subjective
experiencing required. Believing or not believing it has a
subjective experience also not required.

Well, I think that anyone hung up on thinking we have subjective experience
which machines don't already have is going to be so distracted that they
will never grasp the significance of what I'm saying. So, we all know it's
fine to talk about RL learning machines as an AI project and ignore the
subjective experience question, but if you honestly believe that humans
have subjective experience and my networks don't, then you will never take
these systems seriously enough to understand them. You will just assume
that though this might be an interesting AI project, it can't be the really
hard or interesting problems of AI, which is why we have this subjective
experience. If that is what is going on in your brain, you will always
right off this type of approach as just another toy, or basic research AI
problem that's no where near doing anything really exciting.

So instead of understanding why this approach is correct, and why it's
important to solve these problems, you will see it as just a segment of AI
that Curt likes to play with and not until I make the thing do something
that no other AI project as done, will you be able to take it seriously.

I thought, 5 years ago or so, when I started posting here, that more people
would understand enough about the problem of human behavior to be
interested in this type of approach. What I've found however, is that very
few even have a clue about what humans are, and most are lost exploring
ideas that are so far off base that they will never figure anything useful
out for themselves. They will simply have to wait until they see some AI
project acting like a human before they will know which approach was the
right one.

WC:
But if that is true, why is it you believe in the idea that
humans have a special power called subjective experience and
machines don't have this when there is ZERO evidence to
support that believe? Why are you so invested in an idea
that has no foundation?

JC:
The foundation for me is the experience itself. But I don't
see it as a special power. It is what it is like to be a
human machine.

Well sure. I happen to be human and I happen to know what it's like to be
me. :) And I know it's very interesting and magical and seductive. And I
know that all other normal humans act in complex ways that are very similar
to how I act, so it's no hard leap to believe they are having very similar
interesting and magical feelings.

And I know that no machine, like this computer sitting on my desk, acts in
any way close to how humans act. So it's quite trivial to assume it doesn't
feel like I do about my awareness and my existence.

But the question here, is does the machine have awareness, proportional to
it's complexity of behavior? Or does it have none of what we have? Can it
feel pain? Can it see redness? Does it love and hate? Does it have hopes
and fears?

My answer is that what humans call "feelings" is nothing more than our
behavior - how we react to our environment. So for a machine to have
feelings, to have hopes and fears, to feel pain, or see redness, it just
has to have the correct behavior - it must react correctly to it's
environment. It must act as if it's having feelings just like humans act.
And those actions are not just the ones we can see externally, they are
also the reactions that happen internally - what we think about - what
happens in our head.

If you can grasp the importance of this point of view, that humans are what
humans do - and NOTHING ELSE - then you can grasp what I'm talking about.
When we talk about what it's like to "feel pain" what we are talking about
is how our body reacts to the things we call "pain". Think about how you
react to a painful event - like sticking a pin in your arm. Think about
how you react when you do the same thing but your arm is numb so you don't
"feel" the pain. The reaction is very different is it not? The things
that happen in your mind, and how you end up moving your arms and legs is
very different in those two cases. All that is happening there, is that we
are classifying one type of behavior as "feeling pain" and he other type of
behavior as "not feeling pain". we do this for both what happens inside
us, and what we can be seen doing externally. A good actor might be able
to fake the external movements and make someone else think they were in
pain when they were not. But controlling the internal behavior of the
brain (aka the mind) is not so easy to control.

In this view of "humans are what humans do", where is there room for this
stuff you want to call "subjective experience". How, for example, is your
subjective experience not just the stuff that happens in your mind - the
stuff you sense (like the pain) and the stuff you do in response to those
sensations?

If you can understand this, you can understand that all there is here, in
us, and in the machines, is behavior. It's just the motion of atoms. And
the motion of atoms is not so special that there are different types of
motion. At the atomic level, the only difference between a computer, and a
human, is the number of atoms, the types of atoms, and the motions they
take relative to each other.

Building a machine is just a process of collecting together enough atoms of
the right type and putting them in the right place. If you put the right
parts together the right way, you will have a machine that moves in the
same sorts of ways that humans move. And if it moves in the way we call
"feeling pain", then it is feeling pain. That's what "feeling pain" means
to me.

Humans are just machines with sensors and effectors and data processing in
the middle. All our conscious awareness of the environment is in the data
processing that happens in the middle. Our conscious awareness exists only
in the virtual environment of the data processing that is happening in our
brain. If we were fed the right sensory signals by a very advanced
computer, our consciousness awareness would be exactly as it is now. We
wouldn't be able to tell if we were sensing a real Universe, or one created
by a complex computer.

This virtual reality created by the data processing of the brain is the
foundation of what you call your subjective experience - it's the thing you
are describing when you talk about "the experience itself".

But we have a ton of other machines already doing similar data processing
functions so it's hard for me to accept that we are somehow special. We
are atoms, robots are made of atoms - it's all the same "pink" on the real
inside. Our atoms create this interesting virtual experience for us, why
would we believe that a robot is not also having an interesting virtual
experience and if it's an reinforcement trained robot, why it's not also
feeling pain - just not in ways that our as complex as we "feel pain"
because "feel pain" just means - "the complex types of behavior we produce
in response to the stimulus signals we call pain". We don't know pain by
how it feels as much as we know pain by how react it it.

It may be what it is like for a silicon
machine that behaves "consciously". But essentially it
has nothing to do with observing a machine doing intelligent
things regardless of any beliefs about what it is like to
be the machine. The evidence I am talking about doesn't
involve it having or not having a subjective experience.
It is about a physical demonstration of intelligent behavior.

Sure. But that's a cop out. We all know that if some AI project was able
to produce a robot that acted very human in all ways we would all be
fascinated by the project. But before that happens, someone has to figure
out what we need to build, before anyone has ever built it. So we can't
wait for someone else to tell us the answer. We can't wait until someone
shows us the human level AI. We have to figure out for ourselves, what to
build.

Debating what we need to build, is what all my conversations are about -
and my frustrations that others, such as you, haven't figured anything out
for yourself, and are just waiting for someone like me to do all the work,
and show you the answer. And all you contribute to the discussions, are -
"your networks don't work". Long before they start to produce human
behavior, they will be doing interesting stuff because they are on the
right path. But if you can't understand the path, because you haven't
figured out the path for yourself, you will never be able to understand why
these things are interesting even before they start responding to your
Usenet posts.

WC:
So your problem, is that since you have no answer of your
own, you have to look at my ideas as extraordinary since
they are not well excepted by the educated majority.

JC:
I am not sure exactly what ideas you have that you might
think I consider extraordinary? That we are just machines?
That your nets may have "subjective experiences"? No I
don't see that as extraordinary at all.

WC:
And with no ideas of your own, you are forced to wait until I
can give you extraordinary proof that my ideas are valid
(a robot built using my ideas that acted like a human would
do nicely for this - but just seeing the nets do something
you see as extraordinary for a neural network would be good
as well).

JC:
It doesn't have to be extraordinary proof. Just some ordinary
proof would be ok. The kind of proofs used in math and science.

But again, the type of proof you are asking for means you are asking me to
do the work, and show you the answer. I'm asking you to try and understand
the work instead of just waiting for me to give you the answer.

CW:
[...]

This is the stuff that needs to be talked about and worked
on, but I can't get the group past it's obsession with
consciousness and the limits of computation and whether we
need to make the machine out of meat. And until people can
understand and get past these basic freshman level problems,
I've got no one here to chat with about the real problems.

JC:

CW:
Did I leave you speechless here? :)

Not sure, I might have accidentally erased it. You have made it
clear that certain subject matter here has been equivalent to
mediaeval philosophizing about the relations between the four
elements and the four humours but the "real problem" you want
to discuss is not a philosophical issue. Once you have built
the all singing all dancing network then they can philosophize
about what it all might mean :)

I've already done all the philosophizing about what it means. You guys
have forced me to do that because that's what this group wants to talk
about. What I'll be able to do once I've got the hardware dancing, is to
get the rest of you philosophising about what it means at which point you
will catch up to what I've been philosophising about for a long time here.

And I have been working on and off on my own networks although
the nodes have been more like logic gates or neurons than simply
diverting pulses and they have had more brain-like architectures.
And guess what, no consciousness or subjective experiences as
a required part of their description!!

:) Good for you. Are you trying to make them learn something through
:reinforcement? Working on that is where it all gets interesting and hard.
:Though I think the pulse sorting concept has great potential, I'm not
:convinced it's the only, or the best way to solve this problem. The hard
:problem that I think must be solved is creating reinforcement trained
:neural networks that classify the environment like the brain does. Pulse
:sorting networks show great power in being able to be trained, but I'm
:sure other types of networks could be made to do very interesting things
:as well.

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.

Relevant Pages

  • Re: Reinforce learn this
    ... controller, which works as well for the robot, that our brain works for us. ... give us a few years of exposure to an environment with ... Only some basic learning hardware was there at ... This is where reinforcement learning comes in. ...
    (comp.ai.philosophy)
  • Re: The Brain
    ... aka the correct behavior to produce, then the learning ... It has no creativity. ... be entirely due to the environment. ... it must learn to apply that reward to other future situations ...
    (comp.ai.philosophy)
  • Re: Robotics, AI, and Ethics
    ... giving it negative rewards for illegal moves. ... is not how you make the learning algorithm stronger. ... the learner not in the environment. ...
    (comp.ai.philosophy)
  • Re: Robotics, AI, and Ethics
    ... because the one you wrote to play TTT isn't ... I think the "strength" of a learning system is limited ... And if the only reward ... environment we are dealing with, ...
    (comp.ai.philosophy)
  • Re: Is the human brain a optimal generic learning system?
    ... to fit whatever environment they are placed in. ... It can be an optimizer (no generics involved) that ... playing chess, ... And when a programmer is in the learning ...
    (comp.ai.philosophy)