Blank slate learning

ElLoboViejo@xxxxxxxxxx wrote:

(1) I find it odd that so many AI workers seem to believe that
reinforcement learning (conditioning) is exemplified in blank-slate
machines. There is no blank slate. Thus a machine that "learns
arithmetic" must be one that has some pre-existing behaviours which are
shaped to produce the behaviours we call "doing arithmetic."

I'm surprised you think a "blank slate" doesn't have innate behaviors.
There is no blank slate? That's a good one. :) I sure have seen a lot of
blank slates in school. So what do you think a blank slate is? A area of
empty space with nothing in it? I've never seen anyone imply that.

Last time I looked at a real blank slate, it was a physical chunk of slate
which has a collection of innate behaviors such as the default ability to
be uniformly dark, and the ability to change its light reflectivity when it
was marked with chalk and it comes with other features, like a felt eraser.
The slate, the chalk, and the felt eraser are all parts of the device we
call the "blank slate". The starting condition of the slate is "blank"
only because it's initial behavior (all dark) is said to hold no useful
information.

It's absurd to suggest that anything someone refers to as a "blank slate
learning machine" is empty space with no behavior - which is the only way
your comment above would make any sense. Why does everyone forget about
the slate?

Blank slate learning machines are machines which, just like a real blank
slate, start of in a condition with all the memory it uses to hold what it
learns being "blank", or being "all the same value". It has no a priori
knowledge about the environment it must learn in.

In the case of reinforcement learning, what the system learns is the value
of a behavior context association. What the blank slate learning machine
starts out with is a value system where all behaviors in all contexts are
evaluated as being equally valued. A program like TD-Gammon starts with
the innate understanding that all moves in the game are equally good. That
is its blank slate starting condition. It has no innate understanding of
game strategy. But as it learns, this "blank state" staring condition is
transformed into one where all the moves take on different values, just
like what happens when you start to write all over a real blank slate.

"blank slate learning machine" NEVER MEANS "no behaviors". It means "no
knowledge about the environment in which the innate behaviors are to be
used". All learning machines have innate behaviors. A robot with wheels
has the innate behavior of turning the wheels. An ANN had the innate
behavior of producing an output value as a function of its input values.
My blank slate pulse sorting network has the innate behavior of sorting
pulses. What they don't know when they start, is which behavior is better
than another in the environment they will be placed into. It means no a
priori knowledge of the problem it is to solve before it starts learning.
It's knowledge about how to use its innate behaviors is what is blank, not
it's ability to produce behavior.

Blank slate learning machines never learn new behaviors. They simply learn
how to best use their innate behaviors. We like to talk about a behavior
as a "new behavior" when in fact it never is. All that happens, is that
the probability of producing a sequence of innate behaviors changes as it
learns. A reinforcement learning machine could have innate behaviors of
hitting keys on a typewriter. When it starts, the assumption is that any
key is as good as any other key to hit. So it will produce what seems to
be random behavior. Any sequence of behaviors will happen, if you just
give it enough time. Of course, to type the works of Shakespeare would
take billions of years. But with conditioning, you can shape the
probability of the system's behavior to produce it in far less time. That's
how reinforcement learning works. It doesn't "create" new behaviors. It
simply shapes the probability of the innate behaviors that already exist in
the system. But yet, we call that "blank slate" learning because of the
fact that the probabilities, like the real blank slate, are all the same
value when it starts.

Blank slate learning in AI never means "blank behaviors". It means no a
priori knowledge about how to use it's innate behaviors.

In the case of reinforcement learning, there's yet another aspect people
get confused about. They see human behavior as having innate drives or
needs and as such, we must not be "blank". Again, in the case of
reinforcement learning, this is not part of the learning system, but
instead, it's in the critic which is logically seen as part of the
environment, and not part of the blank slate learning machine. However, in
all real world machines, like humans, the critic is implemented as hardware
in us, so it is part of the human, but not part of their "blank slate
learning hardware module".

It's very unclear just how "blank" the brain really is and how much a
priori knowledge might be genetically built in. But, the majority of our
behavior is blank slate learning by the fact that at birth, we have almost
no knowledge about how to use our innate behaviors (arm, leg, foot, hand,
finger, head, eye movements). Other than a collection of very minor (but
important) behaviors like sucking, the value of almost all our adult
behaviors must be learned from experience. Compared to the large and
complex behavior value system we have as adults, a new born human baby is
for all intent and purposes, blank.

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.

Relevant Pages

  • Re: Reply to Wolf
    ... So how did it ever evolve? ... Machines do not have to be seen as models of anything. ... It has to with the system that does the learning. ... Actually I don't think random behaviors are the only movers ...
    (comp.ai.philosophy)
  • Re: hello?
    ... even blanker as it wouldn't need what few innate behaviors ... I just see that as multiple layers of learning on top of one another. ... another layer of evolution learning systems on top of other evolution ... Take the example I've used a lot where you reward a robot for charging it's ...
    (comp.ai.philosophy)
  • Re: How does this robot know it has arms?
    ... that learning without innate ability is impossible. ... behaviors we call language use. ... no use if we don't have a system of selection which has the power to ...
    (comp.robotics.misc)
  • Re: Blank slate learning
    ... All the arguments I've seen make the argument that since we have an innate ... WE don't know _any_ language at birth. ... against the blank slate view. ... Sure, first of all, a large chunk of the brain by volume is not a learning ...
    (comp.ai.philosophy)
  • Re: brain thoughts
    ... and types of behaviors human have. ... to implement reinforcement learning. ... You see such a network in the neocortex, ... to be made for the analysis of the sensory input, ...
    (comp.ai.philosophy)