Re: Brain insights from machine intelligence

casey <jgkjcasey@xxxxxxxxxxxx> wrote:
On Jan 10, 8:05=A0am, c...@xxxxxxxx (Curt Welch) wrote:
...

This is exactly the error you are making in your argument.

I don't think there is any error in my "argument". From what
you have written it seems to me I just talk about it differently
to you. Like Ashby pointed out in section 9/16 "... often the
contradiction is due to the fact that the two arguments are
really referring to two distinct sets, both closely associated
with the same object or organism.

I think it would be helpful to forget about the brain and
restrict ourselves to actual program examples that can be
pointed at so there is no confusion.

Well, lately, we have been debating the nature of the learning problem the
brain solves. We can't forget about the brain completely because it's the
touchstone we are trying to duplicate - though we can forget about it's
implementation when discussing the problem to be solved.

We can simply switch to the problem of programming a robot with arms and
legs and eye as it learns by trial and error how to interact with it's
environment. Or we can create some simplified subset of the sort of problem
- which is still high dimension enough to be the same class of "hard" as
the larger set of problems the brain solves.

... if the network has 1000 bits of data for storage,
it might use 500 bits to represent the last inputs,
250 bits to represent a summery of the previous inputs,
120 bits to represent a summary of the data from 3 time
slices back, etc. So the resolution of the "memory" of
the network fades with time.

Yes, that is how I understood your view even if what I
wrote gave another impression. However your view of a
fixed percentage "fading out" over a fixed time is far
from optimal.

What do you mean by "optimal"?

There are many cognitive experiments where people are given
data of various kinds to see how much they can reproduce
and any changes they have made to the data. Memory in
biological systems is dynamic, selective and processed.

Sure. My networks implement a type of memory which "fades with time" but
yet, any specific point it "remembers" or "fails to remember" is not so
exact as the example I gave above when I talked a bout 250 bits used to
store the last step.

That's because it's highly sensitive to _what_ is being accessed and used
based on context. That's because (as Ashby talks about in his brain book)
that the learning parameters to be adjusted must be selected by context.

The context of the environment controls what part(s) of the brain are
currently being used. When the context changes, that last part (based in
the last context) used is left in it's last state. If we then spend 5
minutes in a totally different context, that old context remains fairly
unchanged. And when that context is reactivated - either direction by the
environment, or indirectly by our "thought process", the last state that
aprt of the brain was in controls the actions - a memory of that past event
is "present" in our brain.

If the context we drift to instead, has large amounts of overlap, then
large amounts of the old context gets lost - making it very dificult to
recreate that previous context without getting it confused with the very
similar, but later contexts.

If I show you one card, and ask you to remember it, I can then talk to you
for hours, about subjects totally unrelated to cards, or numbers, and then
ask you what card I showed you, with a high probability, you will be able
to remember correctly. That's because the "playing card" context sections
of our brain were not disturbed by all the context changes that came after
it.

If however, I repeat the experiment, but in the hour following, I show you
100 card tricks, many of which require you to remember a playing card, the
odds of you being able to remember that first card, is highly reduced.
That's because the "playing" card elements of the brain were constantly
being reused over the following time period, and were in effect overwriting
many of the important associations that were formed in that first context.
The contexts of the different events has large amounts of overlap - large
amounts of the same brain pathways were being reused.

Because of this, if we look at the current state of the entire brain, we
will find each area of the brain will have an effective memory going back
different amounts of time. It will be a function of when that part of the
brain, was last used. What it will create however, is the same effect I
talked about above. If you plot a histogram of each area of that brain,
and when it was last used, you will see a decaying "memory' graph that has
a lot of neurons with very recent use, and the further you go back in time,
the fewer brain areas will still be left that hadn't been used since then.

That I believe is basically how we end up with a memory that works well in
the short term, but fades with time. It doesn't requite that the
individual neurons have some sort of "fade with time" effect. It only
requires that different parts of the brain get activated based on the
current context of the environment and that they have some sort of memory
of the last activation patterns. I do suspect however, that the basic
units do actually includes some sort of fade with time effect as well, but
it's interesting that it's not needed to create a fade-with time memory
effect at the network level.

How far back such a system extends is a function of the
details of how it works. The important aspect however,
is that the short term memory doesn't just drop off a
cliff where its memory goes from good, to none at all,
as it passes some time limit - which is what would happen
for example if you just tried to implement temporal
memory as a simple time delay shift register that stored
all past inputs for X cycles.

It is not really up to you to say what a short term memory
can or cannot do. If it can do it then it can do it. There
is no reason a short term memory can't remember everything
for a fixed time interval and then "drop of the cliff" if
that solves a problem.

Sure. If "the problem" is the one you actually want to solve.

"the problem" we need to solve (generic reinforcement learning in a high
dimension environment), can't be well solved with a short term memory that
"drops off a cliff". That's because memory of the recent past is needed to
correctly establish the current context of the environment since the
current values of the sensory signals are always a very poor representation
of the environment. If we are chasing a rabbit and it vanishes down a
hole, we can try to dig in the dirt to get to the rabit. But once our
short term memory "drops off a cliff" we will have no memory of the rabbit
and as such, no motivation to keep digging. The limit of how long we can
continue to produce a behavior in response to a stimulus (like the rabbit)
will be limited to how long the past events of seeing the rabbit remain in
our current context of the state of the environment (aka rabbit is part of
the state of our local environment).

If the memory works by storing _everything_ up to a point where it ends,
the system will have a very high resolution memory up until it runs out of
memory and the past experience will be totally gone. But if the amount of
memory is used in a decaying fashion, the context can be based on past
events extending far into the past. Where as a memory with no "fade"
effect might remember for 5 seconds, with a fade effect, the same amount of
memory can remember important nightlights about past events for hours.

In the rabbit and digging example, the system doesn't have to remember
every detail about the rabbit to allow the dog to keep digging. It only has
to remember "rabbit food in the ground". The details about where the dog
chased the rabbit, or what color it was, or how many spots it had, can all
be forgotten, but if it is able to remember "rabbit" for an hour instead of
for 5 seconds, it can continue to dig and otherwise search for the rabbit
for an hour.

Likewise, if we put our keys down on the table, and then walk away, we
won't be able to remember where we put the keys after the memory "falls off
a cliff". The location of the keys is part of the context of the
environment defined when we saw them sitting on the table. To understand a
good current state of the environment, we need to keep that memory of the
location of the keys in our memory. If the next day we want to get our
keys, and we haven't trained our sleeves to always leave the keys in the
same place, we won't be able to remember where they were left if our memory
system wastes all it's memory on short term high resolution memory and then
"drops off like a cliff".

The phonological loop is believed
to be such a system unless you can come up with a better
explanation of the observed data.

So now you return to the brain when you said you were going to ignore it?
:)

I don't study much in the way of brain data because it's not the direction
I'm attacking the problem from. I have no clue what data you are talking
about or what you think the "phonological loop" is so without first
studying the experiments, and the data, I can't even suggest a possible
answer.

But any possible answer I would come up with will fall into a few possible
categories. 1) the effect is caused by specialist innate hardware that has
nothing to do with our generic learning skills that makes us intelligent -
and as such - I have little to know interest in it at this point of the
effort, 2) it's an adaptation of the generic learning system created by
evolution to deal with one specific problem - and again, at this point of
the search - I don't care about specific specialized adaptations - we first
have to find the generic learning system before we waste time on
understanding how Evolution adapte4d it to each part of the brain, 3) I'll
make up some bull shit about how it's a possible side effect of what the
generic learning system does.

Either way, the result is that the data is highly unlikely to effect my
opinion that we need to find a strong generic solution to this learning
problem before we worry about anything else in "real AI", and it's likely
not to yield any further understanding of the generic learning problem, or
of potential solutions.

However, if you want to give me a quick, run down on what you are talking
about, I'll be happy to make up a bull shit response! :)

This power the brain is because it's using a better algorithm -
it's using a machine design with the power of a quick sort,
compared to our current learning algorithms that are more like
bubble sorts.

Your bubble sort vs. quick sort analogy is _exactly_ how
I view it as well. But this is not "more of the same" or
the just the result of a "bigger brain" it is a completely
different algorithm which I would suggest had to result
from the evolution of a new circuit or, what I believe
did happen, from a modification of what already existed.

Yes of course. Learning doesn't happen by magic. It happens because
evolution spend a few billion years DESIGNING A DAMN GOOD LEARNING MACHINE.

Designing a better learning machine is not, in any sense, a "simplification
of the _problem_ by innate hardware built by evolution which is otherwise
impossible to solve" which has been you standing position in this debate
and the only position I have taken issue with (in these recent threads).

There are many problems in the world which can be solved by simplification.
We transform, or redefine the problem, in a way that makes it simple to
solve, where before, it was impossible (or at least much harder) to to
solve. For example kn robots, we might try to build a car that can drive
it's. WE want it to keep itself on the road, so we give it a video camera
and try to write software to use the video signal to control the car. We
find this to be so hard that we feel we cna't make it work. So we SIMPLIFY
THE PROBLEM - we decide to paint a stripe on the road, and then put a
stripe-sensing array on the bottom of the car, and then try to write line
following code in the controller. We find this works very well, and our
problem of making the car keep itself on the road has been solved,
SIMPLIFYING THE PROBLEM.

This is a perfectly valid engineering solution for hard problems. When
they are too hard or too expensive to solve, we look for out-of-the-box
solutions to change our goal - to change the problem to something simpler.

But the learning problem the brain solves is well defined not by how the
brain solves it, but simply by the behavior the brain is able to produce.
There is no way to change the fact that the brain is able to produce very
specific types of leaning behavior (which is trivial to test for and
demonstrate using real humans), and that in order to solve AI, we have to
duplicate that learning skill. It's not a problem which can be simplified
by some sort of innate pre-processing that was optimized to the environment
by millions of years of evolution.

It is however, a problem that was solved by evolution building a very good
generic learning machine. And we have to do the same to "solve" AI - build
a learning machine of equal power. We don't have learning machines that
come anywhere near to the learning power of the brain yet and that's how we
know it's so obvious we have a highly important, and big, piece of the AI
puzzle still missing.

I claim that this missing piece is far more than just one piece of the
puzzle. I claim it's THE PUZZLE, and once we have it working, the debate
about whether AI is possible will quickly die out. It will be the turning
point from were everyone debated when, and if the dream of AI could ever be
realized, to the point that everyone understands that AI isn't a dream, but
is real, and here.

The first solutions will just make our little robots act, and look, alive,
and conscious. It will make the machines, act, without mistake, with that
awareness and purpose of action, we see an all animals. They won't be
anywhere near full human intelligent behavior. That will take time to
develop and will require a lot of detailed testing and comparison of human
behavior to AI behavior. But what it will show, without a doubt to most
people, is that AI is real, and that intelligence, is nothing more, than
the type of behavior that naturally emerges from a reinforcement learning
machine. Getting that intelligent behavior to closely mimic the behavior
that actually emerges rom humans, will take a lot more work.

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.

Relevant Pages

  • Re: A reply to Marvin Minsky.
    ... learning algorithm, without such substrate-like support, will do nothing. ... Just as brain uses a myriad of inputs (from ... memory for an example of where Cog. ... experiments should characterize more fully the beneficial impact of emotion ...
    (comp.ai.philosophy)
  • Re: God and science.
    ... consciousness more deeply. ... Synaptic plasticity and the neurobiology of learning and memory. ... Learning and memory are fundamental higher brain functions that allow ...
    (alt.gathering.rainbow)
  • Re: Gobbledegook
    ... Much learning is unconscious or when not 'trying'. ... No, just means unaware, in this context. ... Not any more, my brain has had enough of this, ... "The brain’s exact storage capacity for memories is difficult to calculate" ...
    (uk.legal)
  • Re: Episodic Memory: The Challenge
    ... memory that we had made soup, it would be as if we ... (taken from Phantoms in the Brain - Ramachandran.) ... be correctly regulated since most the low level context data is missing. ... is just a duplication of that part of the experience. ...
    (comp.ai.philosophy)
  • Ive an odd feeling Ive read this before
    ... memory is a reliable tape recorder. ... feelings of familiarity simply should not exist. ... spanned a wide spectrum, including past lives, precognition, brain ... a professor of psychology at Southern Methodist ...
    (uk.philosophy.humanism)