Re: face-selective region


Curt Welch wrote:

..... [continuation of response to prior post] ..........



From there, you have to add a reinforcement learning system to map the
current state (i.e., the stimulus signals), to behavior. So somewhere in
the link from the state defined by the activity in the cortex to behavior,
the brain must be implementing a reinforcement learning system that
strengthens behaviors based on rewards and weakens behaviors based on
punishments.



H does have his associative memory as the 2nd major part of his
hierarchical scheme. As he mentioned, the power of the overall system
comes from having utterly huge amounts of such memory. This of course
deals with the so-called "common-sense" problem. How can a human
successfully function in the range of environments it functions in, and
escape the usual brittleness problem of most AIs? The primary answer is
to have a lot of memory with stores for many different real-world
situations, skills, etc. This is part of the solution, but the weak
part of H's model is tying everything together.

Also, as you indicate, there does need to be some sort of goal-system
overlaid on the rest of it which indicates which items to store and
which items to ignore. Towards the end of the book, H does talk about
applications for his model, but these are somewhat limited apps, such
as air-traffic control, so the goals are really set ahead of time, and
the domain of the learning situation limited, by the external user.

There are a lot of ideas in the book. H indicates that learning can
conceivably take place at "every" level, but this doesn't seem very
true for biolgocial hierarchies.As I've indicated many times,
plasticity seems to increase as you go up the levels, with more
hard-wiring at the lower levels.




Regards which behavior is to be ultimately produced, well, H's model is
as yet incomplete. It's basically a framework for a model, with many
details needing to be filled in. Likewise, I thought it was a little
too weak on temporal aspects, production of multiple sequential
operations, etc. I should probably review these aspects of H.

His idea of how it generates output sequences is that it performs the
inverse of the sensory system. So what needs to happen is that if you
wanted to create a sequence like what was required for the Gettysburg
Address, the high level system needs to first activate the cells which
represent the concept of the Gettysburg Address. Those would activate and
stay active for the entire processes. Those in turn would trigger the next
lower concepts in the sequence, like "Four".



Yeah, I get this. Inverse hierarchy. Divergence of signals going down
the motor hierarchy, analogous to convergence of signals going up the
snesory hierarchy. That's very short-term. You still need a good way to
deal with longer-term activity. Even going around the entire outer
loops [ie, internal comps + external behavior], the loop activity is
very fast, under a second or so. You still need a good way to direct
actions over the course of many seconds, and even minutes.

With feedback systems, one thing leads to another, which leads to
another, on and on, so everything is done on the context of the
previous, but you still need some long-term directional control, else
you'd simply be responding constantly to immediate contingencies.
Brooks' reactiveness extended a bit with STM. It's not clear to me from
H's model how you get more long-term planning into the mix.



That would stay active for as
long as you were saying or writing "Four", and would in turn generate the
next lower level sequence of signals needed to produce the "Four" behavior.
The "Score" behavior would be triggered by the combination of context
signals "Gettysburg Address" and "Four", and maybe even the the lower level
signals for the trailing behavior of producing the "Four" sound. So, the
only temporal signalling needed in the hardware is very short term - just
to create the correct delay from the end of "Four" to the beginning of
"Score". So the hardware which activates the "Score" internal state only
needs a short term temporal relationship to the context that exists just
before we would say "Score" when trying to generate the entire sequence
known to us as the Gettysburg Address.

Each step of a long sequence is generated by the hardware for each step
recognizing that it was now time to become active based on all the context
signals active in the cortex.

Where Jeff seems to see this as the motor cortex doing some inverse of what
the sensory cortex is doing, it doesn't in fact have to do that at all. It
can do the same thing just by acting as another sensory cortex watching
what outputs the lower brain is sending to the muscles. By doing that, the
motor cortex "knows" in effect, the current context of the motor behavior.



Yes, feedback in the local motor pathways, combined with feedback from
outside around the longer sensory loop hierarchies.



The motor cortex is in effect recognizing that we are currently reciting
the Gettysburg address. And using that fact, each next behavior in the
sequence can be triggered, based on how far we have currently advanced in
the processes. So the motor cortex is driving behavior, but it's driving
it by telling the rest of the brain where we are in the processes at each
moment.



This is still relatively short-term. One thing leads to another. The
previous activity goes around the feedback loops [internal and also
external], is filtered, and then input to the associative memory
system, and new associated outputs are generated. That's fairly
straight-forward. That's the power of associative memory combined with
systems for abstraction and feedback. The output of the memory system
ultimately becomes the next input to itself.

However, you still need the value/goal system to oversee the entire
process. We're not simply reactive beings. There's more to life than
that. This seems to be missing from H's model at present. There is one
tiny reference to goal-directed behavior on p 158.



Rememer H had the hippocampus at the "top" of his hierarchy, but I
think this was just a wild shot in the dark, and wasn't too compelling.

Yeah, I wasn't either.



Reading some other stuff [I'll start another thread on this, regards
hierarchy and dynamics, from book called The Primate Visual System, ed
Kaas + Collins, 2004 .... good book, you should take a look], the idea
of the HC at the top of the hierarchy probably comes from van Essen's
diagrams of the 30+ visual areas, with HC on top.

However, I think H misinterpreted this. The diagram shows the HC on
top, because essentially the only visual area it connects to is ER
[enterorhinal], which is shown on the diagram as being atop the
hierarchy of visual areas. The HC is not shown to connect with any of
the 30 other visual areas. However, if true, to me this means the main
control of STM is at the the top, rather than the HC being some kind of
"executive" module.

IOW, I interpret the system as the 30+ areas mainly being hierarchical
filters, and STM doesn't factor in significantly until later [ER level,
which is up approx 13 levels into the hierarchy].



I suspect the correct answer is that the hippocampus is the controller in
the reinforcement learning system. It's the hardware which implements
reinforcement learning. And the reason Jeff couldn't figure out what it's
purpose was is because Jeff hasn't yet grasped that there must be a
reinforcement learning system at work to explain how the brain actually
shapes behavior, instead of just shaping knowledge - which is what his half
of the equation answers.



Yes.


However, with such a massively-interconnected system, you might not
need a top-level executive at all. Rather, recurrent activity in the
feedback loops keeps driving one thing to the next.

Right, as long as you have some type of temporal system that can active
signal X some short but fixed period of time after signals Y and Z happens,
then you have the foundation for a sequence generator. Whether that delay
is created with a recurrent feedback loop or other system is not important.

The output of each
process is the trigger to the next one in sequence,

Exactly.

but at the same
time, the overall activity is continually being refined by the same
recurrent activity. The loop activity is continually stimulating output
from the associative memory, which enters back into the loop activity.

But to create our type of structured behavior, you also need signals to
define long term context - such as the signal that activates, and stays
active, the entire time you are reciting the Gettysburg address.



Yes, as I discovered above, what seems to be a major shortcoming of H's
model.


This is
needed to allow you to say Four Score in that context, and Four Five, when
in the context of reciting your numbers. Without a hierarchy of context
signals to guide the sequence generation, you would be forced to always
follow the word "four" with the same behavior. So you could learn to say 4
5, or four score, but not both.

And that is exactly what the neocortex is doing for us. It creates a
hierarchy of context signals not otherwise present in the raw sensory data.
Where as there is no single signal in the raw sensory data to let the
behavior generation system know there is a cat in front of us, the cortex
learns to spot cats and gives us that "cat" signal so we can generate
different output sequences when there is a cat in front of us than when
there is a dog in front of us.



Well, in real brains, we have parallel pathways from subcortical
centers including the amygdala and other limbic areas which factor in
emotions and other goals/values. Missing from H.



One criticism of behaviorism is that behaviorism doesn't tell us how the
system learns to "see a light" as a stimulus signal. Jeff's theory about
how the cortex operates as an invariant signal generator answers that for
us. You just tack on a reinforcement learning system to create the correct
mapping from invariant signals to output behaviors, and feedback the final
output to the same invariant signal generator to allow the motor cortex to
create invariant signals which describe what we are doing, and there you
have the correct and full framework to answer what the brain is doing.



Hierarchy + associative memory + extensive feedback interconnections.



I think the entire system works just like the example given several
times in the past of the tennis player hitting the ball over the net.
Things just don't start and stop, and decisions made, at "specific"
points in time [like Libet tried to measure in his experiments].
Rather, one thing leads to and blends into the next, and each change in
input and output that occurs in the meanwhile refines the onging
activity. A guy just doesn't hit the ball when it gets to the racket.
Rather, he continually follows the ball visually, moves his body into
position predictively, prepares the racket for where he will move it
to, on and on. There is no one process and one specific time point,
there are many things going on constantly which are interacting with
and modifying each other. Same for tennis, same for language, same for
other sequential activities. This goes back also to Clark's book we
mentioned recently that the brain evolved to control actions.

yeah, exactly. The behavior generated by the brain is just a constant flow
of micro behaviors all triggered by the current context as understood and
decoded by all the signals in the cortex. Ever pulse generated, is the
next micro behavior. That's the level of behavior the brain is working
with. Pulses. Each pulse is the brains next behavior. So, in order to
hit a ball with the racket, the brain must generate millions of pulses
(micro behaviors) all based on the current context of the environment, as
defined by the sum total of all the activity in the neocortex.

And I've now moved on to a solving both problems in each node instead
of having to use two networks to do it. My pulse sorting net attempts
to solve both sides of the problem at once.

His predictive memory approach explains in fairly detailed speculation,
how the brain "knows" what is going on in the environment and how it
"knows" how the world works, but totally fails to say anything about
how the brain uses that knowledge to make behavior decisions.


As mentioned above, H does have a downward directed motor hierarchy
which is complement to the sensory side, along with associational
connections on top of the hierarchy.

Right, he does. Right idea, but wrong implementation. The motor cortex is
not an inverse of the sensory cortex. Motor outputs flow up exactly the
same as sensory data flows up in the cortex.



Maybe you mean motor feedback flows up.



It sounds like your new nets have abandoned a hierarchical scheme
altogether. No?

Hell no I've not abandoned it. The hierarchy is the foundation of our
complexity. Without a hierarchy at work we would not be able to understand
that we were listening to the gettysburg address instead of just listing to
a sequence of words. Without a hierarchy at work, we couldn't produce the
word "four" in two different contexts. We would have to learn the word
once to say it in the sequence "four five six", and learn the behavior of
speaking the word a second time from scratch for the sequence "Four score
and ..".

My network is the hierarchy. Each level of network adds another layer to
the hierarchy. The function of a node in the middle of the network is in
effect "used" by all the upstream nodes that have the power to route data
to it, and in turn, all the downstream nodes that can receive a pulse from
a node in the middle of the network are being "used" by the node. It's
very much a huge hierarchy. And when you add global feedback to create the
motor cortex half of the system, you in effect create a hierarchy of
infinite depth (not to mention that the same thing happens just because of
the feedback though the environment that happens (we listen to ourselves
talk and that creates infinite recursion limited only by time itself).

So, for example, his
theory might explain how a mouse or a human, would recognize part of a
hallway and know before turning the corner, that the cheese is probably
right around the corner, but no where does he seem to answer why the
brain would try to get the cheese, instead of trying to run away from
it.


Oh, goals and emotions. As I recall, H specifically say emotions
shouuld best be left out of this model, in favor of having a completely
rational system [eg, don't want a air traffic-controller AI freaking
out - [aside, ever see Pushing Tin?].

Emotions are the correct answer. That's exactly what a reinforcement system
creates. But without them, you have no purpose, no drive to do anything,
nothing to make you care about eating the cheese instead of smashing it
with a hammer. Without emotions, you have no purpose, and without that,
you have no intelligence. Understanding the world doesn't help if you
don't have something to do with the understanding. You can't build a
rational AI system. That's an oxymoron. If it's not emotional, it ain't
intelligent.



Like expert systems, I think H sees initial applications of his
intelligence model to limited domains, such as air-traffic control,
rather than as human "replacements", ie, able to exist in situations
that generally require so-called "common sense". Therfore, the goals
are essentially instilled into the system by the user.



If you want to build a mechanical reasoning engine, then you can leave out
the emotion, but you also leave out the intelligence and common sense when
you do that. There's a place in the world for such systems, but it's not
real AI.



One step at a time.



The actual brain has several sub-cortical systems that work in
"parallel" to act on the cortex, to do with goals and emotions. Edelman
called them value systems, I believe. I'm not sure what are H's ideas
on this.

Yeah, emotions, and values are all the same. The heart of all reinforcment
learning system is value calculation. It must calcuate a value fuction
which is used to select beahviors. It's what allows the system to pick one
beahvior over another. It's what causes the mouse to eat the cheese
instead of ignoring it.



The nice thing about the H framework is that it's easy to add parallel
systems that can have totally different architectures [as the
subcortical and limbic areas have], and wire them [both as inputs and
outputs] into appropriate points in the hierachical framework.

Eg, sample ongoing activity in general, do some calculations, and then
feed either excitatory or inhibitory inputs back into the hierarchy at
strategic points. Such pathways are found in the real brain, but not in
H's current model.



Or,
even if it for some reason "wanted" the cheese, now the brain would
manage to learn the correct sequence of outputs to get the cheese into
it's stomach. There's where you need a reinforcement learning system.


You have your system here embodied and interacting with the
environment. H doesn't seem to really consider this, I don't think. Not
sure.

I'm sure he considers it. But mostly, his ideas are about how the system
understands the sensory data and he just hasn't talked much about how it
learns to prefer one behavior over another which like I say, is the real
problem of AI. It must "value" one behavior over another using some
mechanical system. He explains how important memory and prediction is to
understand how the world works, both for sensing and behavior, but he just
never seems to talk about how he expects his system to value it's
behaviors. So it understands that its hand will fall off if you whack it
with a butchers knife, but what makes the brain decide to not hack off its
own hand? He just doesn't seem to answer that most important of questions
with his theory.

I've not finished the book so maybe he talks about this later but I
seem to already be past his technical description of his theory.

So, it looks like he's on what I see as the right track, but has only
so far, addressed half of the problem.


Well, I think we all 3 agree on this, H being the third party.

yeah.

And of course, what I like about his theory is that he, like I, believe
there is some fundamental learning algorithm at work for the entire
cortex, and that specialized sections of the cortex form more because
of what data is sent to it, then the way the DNA built it.


Well, I think these last 2 are one and the same.

Sure, because it's the DNA that determines what data is sent to which part
of the cortex. The DNA has the upper hand in the end. But what this tells
us that that the algorithm we have to build is one and the same for the
entire cortex, and the topology is what we have to get right to create all
the different modules that are needed to equal human processing skills.

There's also no doubt that the DNA has also made optimizations to different
parts of the cortex to maximize it's performance for each type of data
processing that happens in the section.

The DNA just decides what
data gets sent to which part of the cortex, (i.e., where each sensory
signal is connected to the cortex and maybe how different sections of
the cortex are bulk-wired to other sections of the cortex) then the
function formed in each section is a result of the characteristics of
the data that shows up there. So the DNA defines the network topology,
and the generic learning hardware which is the function of the cortex,
auto configures itself to understand the type of data it is dealing
with.


Well, of course, this last bit is one of the big mysteries. ;-)

It can't be quite as loosey-goosey as you say here, else you wouldn't
have the same areas doing the same things in every different member of
the species.

Sure you would. The mammals all share common ancestors so they also share
common design traits. If you hook the optic nerve up to the back of the
brain in a mouse and in a human, you would expect find the visual centers
in the back of the brain in both a mouse and a human.

If mice and humans tend to look at the same world, with a similar type of
eye, then you would expect the data flowing to the visual cortex to be very
similar in nature as well and that would lead to very similar hierarchies
of understanding developing (both human words and mice words have lots of
constant objects generating consistent visual patterns to be recognized).



Yeah, my point was that, this is so because the overall connection
scheme is dictated by the DNA. Learning doesn't work in a vacuum. It
works in the context of a system highly-structured genetics and refined
by evolution.

.

Relevant Pages

  • Re: face-selective region
    ... cortex will become active and stay active ... using a reinforcement learning algorithm. ... AI problem - how does the brain decide what behavior to produce. ... Rememer H had the hippocampus at the "top" of his hierarchy, ...
    (comp.ai.philosophy)
  • Re: We Robot
    ... Consciousness without a cerebral cortex: ... that consciousness is actually in the gray matter of the brain as it were.. ...
    (talk.origins)
  • Re: Strong AI Thesis (No Chinese room, I promise)
    ... The result showed the part of the cortex ... This is why whey they do brain ... The lower brain is not generic learning hardware. ... The higher level behaviors are more variable than the lower ...
    (comp.ai.philosophy)
  • Re: Neural netss (was Re: death of the mind.)
    ... its 30+ visual areas and 1100+ feedforward and feedback pathways, ... This whole thing makes sense, regards the cortex, if you think about ... feedback signals from the other 29 areas, ... they filter and perform the basic ...
    (sci.cognitive)
  • Re: exhaustive definition in the life sciences - an oxymoron?
    ... >Both pathways are part of the cerebral cortex, ... >a 3rd central visual pathway, but it has largely been overtaken by the ... the act of thinking) but does this mean that same signals ... Retina-muscle feedback signals carry depth cues. ...
    (comp.ai.philosophy)
Loading