Re: Molecules and Neurons - and photons

focus2000x@xxxxxxxxx wrote:
Curt Welch wrote:

OK, after this let me go to the point of your network
solution as well as Jeff's Hawkins. I don't think it has
a chance of working. One of the biggest problem is lack
of the STM functionality. What the STM has to do with
all this stuff? Well it has a lot and is very important.
From this perspective Jeff's system has the same
problem as yours and in my opinion no chance.
To be more specific, just ask yourself what is STM
and what is the purpose of STM in our brain.
There are many more problems but just this one is
sufficient enough to render the whole model unworkable.

Both my networks and Jeff's have short term memory (I assume STM means
short term memory - if not, everything I'm about to write can be ignored...
:).

What I'm trying to build with my networks, is a machine that produce
reactions to the current context of the environment. But the "context" can
not be defined by current sensory inputs alone. It could if the inputs
have the mark of property and gave the agent a 100% accurate picture of the
total environment (which is typical for RL algorithms applied to toy
environments - but not possible in the real environment). So, in order to
produce a better internal representation of the current context which it
should react to, the system must define the context based on some
combination of current and recent past inputs. And that's where STM comes
into the picture. The system must have a memory of recent past events so
that it's current behavior, can be a function of recent past events.

Ideally, it would produce reactions based on all past inputs - but that of
course is never practical, so it must use as much of the recent past inputs
as is possible, and it must use some system to allocate data to it's finite
short term memory.

In my network, every node in the network has short term memory built into.
Each node remembers the last pulse to pass through the node - it remembers
when it happened (temporal STM). The STM of the entire network is simply
the combination of all this memory. Though the memory of a single node is
very small, the combined STM of the entire network made up of millions or
billions of these simple nodes, becomes substantial.

In addition, the new design I've been exploring uses a pulse sorting
algorithm that includes feedback from down stream nodes back to upstream
nodes. This extends the temporal length of the STM that already existed in
the nodes because once a pulse is sorted down a given path, it can cause
other pulses to follow. In effect, once a pulse is sorted down a path, the
network had determined that some feature currently exists in the
environment (say a dog to use a very high level concept). With the feed
back, the probability that other pulses will be classified the same (as a
dog) increases because the system already has a short term memory (lasting
for the length of a single pulse) that it just saw a dog. And if it just
saw a dog, then some small feature in the network is more likely to be
classified as "more dog". So once it "sees" a dog, it's likely to keep
classifying following pulses as "more dog". This extends the STM memory of
DOG from the single pulse (built into the node) to many pulses.

So yes, I agree, STM memory is very important. It is what triggers our
behavior. We react not only to the current environment, but instead, we
react to what is currently in our STM - we react to what has recently
happened.

Now, the way I'm attempting to build a reaction machine with STM with my
pulse sorting networks may or not turn out to be very useful, but I do very
much believe you can produce a temporal reaction machine without STM, and I
have very much included it.

Now, Jeff's system also includes STM in a slightly more round about way -
but the end result is very similar. It performs sequential temporal
pattern matching. It creates invariant representation by performing
temporal patten prediction. For a simple abstract example, if his network
sees the temporal pattern A B C D repeated, his network gives is a name
"DOG", and it's output is transformed from the A B C D raw data, to DOG DOG
DOG DOG as the invariant representation. Indirectly, this means the
system, when it sees the D input, and knows it's been tracking the DOG
pattern, has a STM of A B C. It has a STM that tells it there is a "dog"
in the environment.

The lower levels of the hierarchy have a very short memory, where as higher
levels of the hierarchy, will have longer memories because as you get
higher in the hierarchy, the system responds to large, more complex causes,
which tend to be more persistent in the environment.

I don't remember how obvious the STM was in his description from the book,
but it's very obvious in the numental white paper that outlines the actual
technology they are working on.

Secondly, I'm very pleased to notice the inclusion
of RL functionalities. However, the model you are
trying to apply will fail for the same reason for which
GOFAI failed. I can just add to this that RL models
described in books are simply laughable forget
about them and start from scratch.

The standard ones in books are only able to solve toy problems with small
state spaces and where the sensory inputs have the Markov property - they
tell the algorithm the exact and complete current state of the environment.
Those types of algorithms can't solve any of the interesting problems which
all must deal with sensory signals that give only spatial awareness of the
environment and where the environments are way too large to be represented
as individual states in the algorithm. All my work is looking at how we
can solve the interesting problems.

One of the big difference from the simple book RL algorithms and the hard
problems, is that book algorithms typically don't need, and don't use, STM.
To solve the hard problems with huge state spaces and partial sensory
awareness, you have to include STM to create a better context for the
system to create the best understanding possible of the current state of
the environment given the limitations.

A lot of what Jeff's ideas are about, is how such a system will go about
compressing as much state information as possible into a given size STM by
using both spatial and temporal prediction. But as I said, what he seems
to fail to do, is give the machine a purpose so it has something useful to
do with the information once it has gathered it.

Finally, you may ask yourself what that guy is
talking about, you may think I'm a crackpot, it's up to you

:) lots of crack pots here. I'm one of the biggest! Just ask anyone! I
:even just had my junior cadet behavior club card taken away!

So now that I've told you want I think STM is for, and why we have it, what
do you think it's for and how do you suggest it be implemented in an RL
algorithm?

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.