Re: How does this robot know it has arms?

RMDumse <rmd@xxxxxxxxxxxxx> wrote:
On Sep 2, 7:37 pm, c...@xxxxxxxx (Curt Welch) wrote:
But I'm not really interested in the fact that babies are born with
simplistic innate behaviors if they are not needed to understand our
general powers of intelligent behavior.

I understand you are not interested, but I am suggesting perhaps you
should be.

Always a possibility. :)

I know you have a bias to find a way from tabala rosa to learning.

There's great confusion about the concept of tabula rasa. Many would argue
that learning without innate ability is impossible. That's just their
inability to understand that all learning systems have an innate ability to
learn. They choose to look at the system's innate ability to learn, as if
it were pre-wired knowledge, and then declare it's not tabula rasa learning
since the the pre-wired ability to learn means it's not a blank. It's a
stupid game of semantics.

All learning systems are tabula rasa learning because they all start with
the innate ability to learn, then then they add to that, the knowledge
which they learn.

Tabula rasa learning is not something I need to "find". Every learning
system I've built in the past 25 years was already a tabula rasa learning
system.

What I need to uncover, is the type of learning system needed to duplicate
human level learning ability.

I
don't think intelligence works that way. I think intelligence has to
have a pretty strong base of hardwired knowledge, before any thing
made of soft learning can take hold. For example, you can't write an
article for Encyclepida Britania without an innate ability to use
language, let alone many other fundamentals, such as an alphabet of
some kind, a grammar of some kind, and so on. Some will be innate,
some learned. I don't think they can all be learned and none innate.

Exactly. A learning system that doesn't have the innate ability to learn
to use a language, will never learn to use a language. The problem here is
to uncover what type of learning hardware is needed for the class of
behaviors we call language use.

If I want to build a robot that can learn to navigate a maze, it has to
have the innate ability to move through a maze. If it's not born with the
innate ability to move, it won't be able to learn a maze. But to learn a
path through a maze, it needs more than the ability to move. It needs at a
minimum, the ability to sense when it's solved the maze correctly, and it's
likely going to need sensors to help it perform navigation tasks. At the
same time, it's going to need some systems that allow it to learn from
experience so that after lots of time moving around the maze, it has to be
able to use that experience to direct future actions, so that hopefully, it
will be able to travel from the start, to the end, without making any wrong
turns.

Even with all that pre-wired knowledge about movement, and sensing maze
walls, and location in a 2D maze space, the machine is still a tabula rasa
learning system because when it starts, it knows nothing about the maze
it's trying to learn. When it comes to the first turn, it knows nothing
about whether turning left, or turning right, is likely to be the right
choice.

All learning systems work like that. They have innate ability to learn
something, but when they start, they have not yet learned anything.

So, when trying to build a learning system, we must always answer the
question of what type of hardware does the learning system start with -
what are the basic primitives which must be built in as innate ability, and
what must it learn? The maze learning robot will likely have basic
primitives of moving forward, or turning, and the ability to sense walls
blocking it's motion and or sense when an attempt to move, or turn, has
failed, because it hit a wall, and it will have the power to learn a
sequence of these sorts of basic primitives.

But what type of basic primitives do we need to build into a machine to
produce human level behaviors? And what does the system learn?

You seem to be suggesting that we might need a basic primitive such as the
ability to crawl (or at least some basic crawl-like motions which will
later be refined through learning into a productive action). It can
certainly be done that way. For example, we can build a robot with legs,
and write code that makes it move the legs in some fixed pattern that makes
it walk. But then, because our walking gait isn't all that great (it makes
the robot move forward, but the legs are slipping and sliding and on a
rough surface there is much grinding of the gears in the servos because the
gate is wasting energy causing the legs to push in unproductive
directions). So, on top of this innate walking gait, we could add a
learning system to attempt to make small changes to the sequence to
optimize the gait to be more productive. But, how do we do this? How does
the learning system know when it tries some change to the gait, if the
change made the gait better or worse? We have to create some system to
measure success, and use that measure, to guide the learning process.

Learning after all is just a change in function. But random change is of
no use if we don't have a system of selection which has the power to
evaluate success. Something has to be assigning value to the changes so
that the purpose of learning is to actually make an improvement in
behavior. The entire concept of "improvement" implies you have the ability
to assign value to different behaviors.

So, for the robot, we have to add some sort of measure of "better". For a
walking gait, we could use speed as a measure. Or we could attempt to
minimize current draw for a given speed by trying to maximize speed per
power consumed.

None the less, once you have motivation defined (the value function that
the learning system is tryig to maximize) and the primitives that define
the search space of the learning function (leg motions), then what's the
point of having a pre-wired gait in the system to start with? If you have
a good learning algorithm, why not let it start with no pre-wired motion
sequences and let it learn to move on it's own? The goal of moving forward
as fast as possible is the same either way.

The only advantage of starting it with a pre-wired gait sequence, is to
reduce the amount of time it takes to find the optimal gait. Now,
depending on the quality of the learning system, this time can be
significant because as the size of the search space grows, the amount of
time to search it with a learning algorithm can grow exponentially. So
where as optimizing a bad gait might take hours, learning a gait from
scratch might take months or even years depending on the quality of the
learning algorithm.

In the case of living organisms, learning time is an important survival
factor. They might die before they learn to walk. So it's not surprising
that animals are born with lots of pre-wired behaviors, or that humans are
born with various amount of pre-wired behaviors.

But for the most part, the pre-wired behaviors just aren't interesting, or
important. Do you really think that the fact that a baby might start with
a few simplistic behaviors like a drop reflex or basic crawl motions is the
key to how humans were able to invent language and invent space ships to
fly us to the moon? Those low level initial behaviors are there only to
help us stay alive after birth. Our AI doesn't need help staying alive.
We will keep it alive as it learns. If you can learn to build space ships,
and solve problems like AI the robot is surely going to also be smart
enough to learn to crawl without it being pre-wire into the machine.

Some low level behaviors, which act as the starting point of the learning
system, have to be there. But at the level I approach the problem from
those starting behaviors are extremely low level and simple - pulse sorting
decisions in a generic signal processing network.

For example, if we could remove or
disable the mechanism that creates the Moro reflex from a human baby,
do you suspect it would prevent them from developing normal human
intelligence? I don't believe it would.

Actually, the reason these reflexes are known/important is exactly
what you doubt. Doctors check these when babies are born to see if
they are present. If not it is an indication of a defective human
being, who will have developmental problems.

That has nothing to do with what I doubt. If a human baby is born without
a normal reflex, the odds are its got problems far worse than simply not
having that one reflex - it's probably got serious defects in the CNS.
They don't test for the reflex because they believe the reflex is
important. They test for because they believe a lack of the reflect is a
good indicator of much worse problems.

... personally because I'm a lot more interested in figuring out how to
build smart machines than I am in building things that act like humans.

I hear you. But. Baby - bathwater.

We have one example of advanced intelligence. I am suggesting we see
how it comes to be different from other life forms and use it as a
guide to acheiving intelligence.

There's nothing wrong with that approach. But in my study of learning
systems I figured out many things which allows me to make educated guesses
way beyond the need to duplicate every little biological wart humans have
in order to create intelligence.

These insignificant behaviors humans start with at birth are nothing
compared the enormous set of interesting behaviors we find in an adult. We
can walk, and drink, and catch a ball, and cook food, and read a text book,
and put together a book shelf, and design space ships, and program
computers, and tie our shoes. There are billions and billions of different
behaviors a normal human can perform, and every one of them was learned,
not built into our hardware at birth.

What is built into our hardware at birth, is the ability of the learning
part of the brain to receive sensor data from many different sensors, and
control the motion of our arms and legs though it's output signals.
Everything else, between there, are circuits that get configured after
birth to allow us to do all the things we do. How those circuits get
configured by the learning systems in our brain is the key to how we become
intelligent through a life time of interacting with a a complex
environment.

The key to solving this problem is understand what type of circuits the
learning part of the brain is made up of, and what type of learning
algorithms are at work shaping those circuits.

Is the motivation to stay upright, while trying to use the leg motions
that worked for crawling which is causing it to make those first
awkward attempts at a step?

Seems crawling is a hardwired reflex. It was listed as one babys
display on birth if put on their stomachs. They may not be strong
enough but held up, the legs and arms go.

Interesting. But again, not very important for solving the problem of how
we learn to design and build space ships.

But does the Baby look at it's leg while trying to do
this? If so, I think we need to add more complexity to the idea of what
might be happening to explain why it looks at it's leg while doing
this. If it's not normal for the baby to look at its leg as if it were
trying to make it take a step, then maybe the more simple ideas of
trying to stay upright combined with trying to craw forward is what
leads to the awkward attempts at the the first steps. ???

Also the step reflex is innate. It can be detected just after birth.
It's not a full walk, only the tendency to move one leg forward when
both are touched by a flat surface.

Again, interesting.

To the best of my knowlege, babies do not look at their feet when they
walk.

Sounds like a stretch to me. You don't need to assume a genetic bias
of "upness" to explain why we like to be picked up.

True, but there is such a preponderance of the bias for up-ness once
you start looking for it, it shows up everywhere. Even in every day
expressions. For example, take the phrase: higher power. Why not
deeper power? wider power? longer power? etc. Also why do you know
which is better between a lower power and a higher power. There's a
bias, subfusing the word useage.

Yes, I agree. There's a clear bias. Also, if you notice, we only elect
tall presidents. Tall people naturally get more respect. Where does that
bias come from? I think it comes from the fact that kids are small and
parents are tall. We spend our childhood learning that the tall people
have all the power and the short people have none. I think that bias
sticks with us for life. Even after growing up, we learn that tall people
generally have the edge on shorter people for any physical conflict.

In addition, we live in a world of gravity. Height means higher potential
energy which translates to real power. This translates to the high ground
being an advantage in any battle. Just the simple act of being knocked to
the ground means you are at a read disadvantage in a battle because it
gives the other guy the high ground - he can hit you a lot harder by
throwing a punch or swinging a club with gravity than you can hit him swing
your arm against gravity. The high ground is power because of gravity.

We see this "high-ground" effect translated into all sort of human
behaviors. We bow to a person who we want to show respect for - giving
them the high ground. We try to stand tall to intimidate someone and show
them our power. When we draw an org chart, the guy in the company with the
most power is put at the top of the chart. The guy with the most power
gets the highest location in the building (the top floor). And of course,
the guy with the most power (God) is placed in the sky above us all.

Between the effects of gravity and the conditioning we get as kids to
respect height, I don't think you need anything else to understand why
there's a clear bias of height being associated with power in humans.

There could still be some built in innate feature of humans that make us
associate height with power. But again, I don't see that it's needed -
there are plenty of things from the environment that explain it. The only
innate power we need to put into our robots, is strong learning. 99.9% of
everything we see humans do that we label as intelligent, is a learned
behavior - they didn't have it birth. The one innate thing they have at
birth that makes humans intelligent, is the ability to learn all this
complex behavior in only a few decades of training.

I'm not interested in the human "warts" because I know the number one thing
that's missing in our robots right now is strong learning. It's very easy
to program a little startle reaction into a robot and make it look very
human-like. But doing that won't give the robot the power to figure out,
on its own, how to get to the moon, like humans did. To do that, we have
to figure out how to add strong, generic, real time, learning to our
robots. Once we get a handle on the learning problem, then we can look at
the little warts to see what might be needed to make our robots act even
more like humans.

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.

Relevant Pages

  • Re: Ben G on reinforcement-learning and the wirehead problem
    ... Walking at birth is an obvious survival advantage and it's ... Most of their walking is innate, ... it selects to inhibit those behaviors while other behaviors ... Intelligence in the form of generic reinforcement trained learning creates ...
    (comp.ai.philosophy)
  • Re: Ben G on reinforcement-learning and the wirehead problem
    ... Yes, all human behaviors like driving cars, playing chess, ... and so on are learned but it is the innate abilities you can't ... will be of use to the generic learning hardware. ...
    (comp.ai.philosophy)
  • Re: logic from a neural network
    ... and innate behaviors, I'm only talking about the issues ... modules using it wether it was created by learning, ... CPG behaviors using a generic learning system and reinforcement ...
    (comp.ai.philosophy)
  • Re: A current list of what AI cannot yet do?
    ... It's a high level algorithm of a particular type. ... how that is translated down to all the micro behaviors. ... And as TD-Gammon showed we can build our learning ... behaviors on top of that of that innate crap. ...
    (comp.ai.philosophy)
  • Re: Ben G on reinforcement-learning and the wirehead problem
    ... generic and adapted to whatever sensory environment the human was in, ... If the organism is making heavy use of learning already, ... Sucking and swallowing are highly innate for obvious ... what _its_ particular sensory system will do. ...
    (comp.ai.philosophy)