Re: Fundamental Limits to Curt's Nets

Michael Olea <oleaj@xxxxxxxxxxxxx> wrote:
> Curt Welch wrote:
>
> >
> >> Just some stuff to think about...
> >
> > Yes, it is...
> >
> > Thanks for giving me the excuse to think and write.
>
> Thanks for the detailed reply, Curt. You are right - I was unaware of the
> output generator feeding back into the feedforward learner. I'm not sure
> it make much difference, though -- the limits I was talking about are
> limits to what a feedforward scheme can learn. In so far as learning is
> restricted to a feedforward subnet I suspect those limits still apply -
> such nets impose a severe restriction on the space of input to output
> maps. But I will have to reserve comment for a time when I can give it
> more thought.

The question as to what it can learn is a qood one and the most important
question to all my ideas. Just because the net has feedback doesn't mean
it can learn to use it in the ways it needs to.

However, my feed-forward net is very different than most nets which are
called "feed-forward" because it's got memory. So it's not a simple value
calculating function which calculates the current output based only on the
current input. The temporal memory of each node means that the current
output is potentially a function of all past inputs. The learning function
then adds even more memory on top of that.

When you add feedback of all output values, that means it's also got access
to all past outputs. So, each new output is calcuated as a function off
all past inputs and all past outputs. I'm quite sure that gives the system
all the power it needs to create human behavior and to implement any
function you care to try and specify (given a large enough net).

But, the learning power is a question of the networks ability to converge
on the required answer. And though I know the network can converage on
simple answers to simple problems (connect input A to output X) with only
reinforcement learning, I don't yet know the full extent of what it can
converge on. There may be a major class of network configurations that the
learning system could never find on it's own.

So, the part I'm working on now is the most important part - the details of
the learning algorithm, which needs to include careful study and research
into it's power of convergence. Idealy, I would like to find mathematical
proof as to it's power to converge for any learning algorithm I experment
with. But I have to have a specific algorthm before I can do that, and I'm
still struggling with finding the correct conceptualization of how
secondary reinforcement should be implemented.

So, I believe I have a framework that on the surface, has all the power it
needs in terms of what functions it can implement. And it seems to be
idealy suited for the problem of reinforcement learning. Just finding a
workable framework for solving the complex behavior learning problem was
much harder than I ever supsected it would be when I started. That's
because specifying the design of a complex real time recurrent function has
too many degrees of freedom. You can't train 100 different degrees of
freedom with reinforcement learning, so you have to find a way to map all
the degrees of freedom, down to one degree of freedom before you can train
it.

The exicting thing for me is that this network does that. It's a framework
that has only one degree of freedom to be trained for each node, yet seems
to have the structure to produce any needed function. And at the same
time, seems on the surface to have the correct structure to allow it to
find and converge on any answer.

I just have to figure out if the potential I think it has is real or not.
And that's just going to take more work so I can get past this next
conceptualization problem.

> -- Michael

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.

Relevant Pages

  • Re: RECYCLING KILLS
    ... shar-on wrote: ... instead of *less* romantic and with endless enthusiasm for learning, ... I know sometimes i can be somewhat clever, yet at this stage in my life ... I've heard it said that money is freedom ...
    (alt.gathering.rainbow)
  • Re: It doesnt get much clearer than this.
    ... I can't carry due to Canadian law, so I'm learning about freedoms that I ... How ever else you quantify it, it's a lack of freedom of choice. ... some asshole with a gun manages to sneak in and screw up a good gig. ... I often carry a big ugly knife. ...
    (rec.scuba)
  • Re: Fundamental Limits to Curts Nets
    ... the learning power is a question of the ... And though I know the network can ... > decision tree that sorts not pulses but gaps. ... >> of freedom with reinforcement learning, ...
    (comp.ai.philosophy)
  • Re: It doesnt get much clearer than this.
    ... You're learning. ... How ever else you quantify it, it's a lack of freedom of choice. ... If it were freedom, he'd have the choice to do so or not, as he saw fit. ... We have, ad nauseum, but you just can't seem to get it. ...
    (rec.scuba)
  • Re: How does this robot know it has arms?
    ... that learning without innate ability is impossible. ... behaviors we call language use. ... no use if we don't have a system of selection which has the power to ...
    (comp.robotics.misc)