Re: Where is behavior AI now?

"Randy M. Dumse" <rmd@xxxxxxxxxxxxx> wrote:
"dpa" <dpa@xxxxxxxxxxxxxxx> wrote in message
news:1156559944.538305.34330@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Does this post relate to your thoughts last march?

<http://list.dprg.org/archive/2006-March/029764.html>

Relate, yes. However, I was apparently cycling back to the same
conclusion via another route, not having that wonder series of exchange
in mind. But perhaps missing the quality of such discussions.

This time I was more interested in filling the gap between 1991 (or so)
and 2006.

So here's the route I was coming this time. Say you have a simple robot.
You know, our simple robots of today hard mostly hardware, extremely
hardware, here I mean, almost no moving parts, no body joints, no
waists, no heads that swivel, seldom knees or elbows, or articulated
finger tips, etc. Mostly hard materials stuck together.

Take jBot as an example. Two motors. That's it. That's the total number
of (significant) outputs. (Yes there's a display you might be able to
read if you chased it down, picked it up, and held it close to your
face, but otherwise, as far as actuators that deal with the world, two
motors, that's it. Likewise there are limited sensors. 5 (or 4 or 6 I
don't exactly remember) sonars are the primary world sensors. Really we
could stop there.

Yes there are stall sensors, but they don't measure the world, they
measure the motors.

Yeah, but to the control unit (the brain) you should normally look at the
motors as being part of the external world. So if you have some type of
sensory feedback from the motor, then that really is just another world
sensor.

And of course, if you have a stall sensor, it's telling you things about
the external world indirectly - like the fact you have run into something
which is preventing the wheels from turning.

Yes there are navigational sensors, encoders, IMU
with compass, accelerometers, gyros, gps, but they again monitor the
motors and body of the robot relative to the world, and not the external
world for details to make decisions about the robot. To wit, the compass
doesn't measure the magnetic field to see if there is one, and it's
direction and strength, rather it assumes a constant magnetic field, and
then tries to determine the robots orientation to that existant field.

So to my point from this. How many different behaviors can jBot have?

Infinite. But forced to be limited by the finite complexity of the
machine.

It
has two outputs. It has 1 bank (of 5) rangers to sense the world with.
All the remaining sensors are feedback on what the robot is doing
relative to the world, rather than what the world is like external to
the robot. How many different behaviors (evidenced by output actions)
can jBot have?

1) Left stop, right stop? 2) Left forward, right stop? 3) Left forward,
right forward? 4) ...

How many inputs can those behaviors sense?

Infinite. But again, forced to be limited by the finite complexity of the
machine.

Isn't there a limit to the number of interconnections possible?

No. Except for the finite complexity of the machine you build.

Isn't there a limit to intelligence given that matrix of combinations?

No. Again. Only limited by the complexity of the machine you build.

While we can come up with many names for all sorts of fancy behaviors,
isn't there a limit to how much intelligence can be evidenced with BBR?

A lack of sensors didn't seem to limit Helen Keller's intelligence. It
only slowed down her learning about the environment.

Isn't the intelligence of the robot, not in the behaviors at all (which
given a list of all outputs, can be fully described by the states of all
possible outputs), but in the number of ways we can combine those
behaviors to have "emergent" properties?

Yes. But there's a better way to understand the problem. You have to
think of it as a temporal problem, and not a spatial problem. I'll explain
more below.

And there I come full circle again, back to the same place as the March
discussion. Isn't intelligence not in the behaviors themselves, but in
the way they are interconnected, sensed and triggered?

Yes I think so.

In the context of AI, I actually think of a behavior not just as an output,
or an output sequence, but as a reaction. An output sequence (spin the
right wheel clockwise for 1 second) is as you say, not intelligence. The
intelligence comes from when the system chooses to generate that output.
The output alone of is of no importance. Only when it's pared with a
system to trigger the output, does it become intelligence, so only then, do
I think of it as a "behavior" instead of just an output.

But, true intelligence, is even more than a complex set of behaviors built
into a machine. It's the ability of a machine to learn new behaviors on
it's own. The main difference between our current human intelligence, and
machine intelligence, is that most machines we build don't include much in
the way of learning. Most the machines we build we intentionly don't make
them learning machines because they are generally more useful to us as
tools if their behavior isn't constantly changing on us. They are easier
to work with, and control, if we can predict their behavior than if their
behavior is constantly adapting and changing. This is why most people
don't see machines as intelligent (or why many think machines can't be
intelligent). It's because the machines they know about aren't learning
machines.

Now, back to what I was saying about about the temporal (time based) nature
of the problem.

We are commonly taught to think of the operation of complex machines
(computers for example) in the spatial domain. We are taught to think of
them as state machines, which, at any one point in time, the machine will
be in some fixed state. The input sensors all have one fixed value at this
point in time, the outputs all have some fixed value at this point in time,
and the environment, is in some fixed configuration at this point in time
(which with luck the sensors will report to us). The way we design
machines, or write computer software, is often, to think of the machine in
this way, and to specify by hand, the transitions it makes from state to
state.

For an input output machine like a typical robot with sensors and
effectors, we like to think of it's behavior as a function, which maps the
current inputs, to the current outputs, at a single point in time. And for
very simple problems, that can work fairly well. But, for anything
interesting, the machine has to solve problems in the temporal domain.
Which simply means, the current outputs, must be calculated not from just
the current inputs, but from past inputs as well. This means the full
current "state" on which the machine is basing it's decision about what
output to create, is a function of some amount of historic sensory data as
well.

For a simple example. If you have a sensor that tells you wheel position,
the current input only tells you the wheel's position. But, if the machine
has temporal knowledge of the last position, it can calculate average
speed, so the machine can "known" if the wheel is turning. Or if it has
temporal knowledge of two recent past positions, it can calculate change in
speed (acceleration). This is the purpose of a PID controller. It
accumulates historic sensory information and creates an output function
which is not just based on current sensory data (P), but on a summary of
historic sensory data as well (I the summation, and D, the difference).

Lets look at this from another perspective. Lets say we have a very simple
machine with a two binary input sensors, like bumper switches on the right
front and left front of the bot. Lets say the code samples the state of
these two switches 10 times per second. So each, 1/10 of a second, there
is a two bits of binary input flowing into the machine that must create all
the machine's "intelligence" (what outputs it creates to react to the
environment).

You can write fairly simple code that will do something trivial like back
up and turn to the left when the right bumper switch is activated, or the
opposite with the left bumper switch is activated. So the switches are
just acting as triggers to some simple behaviors - a typical bot
programming technique. But there is a lot more information available in
those sensory inputs if the machine is able to track, and respond, to a
recent past history of bumper inputs. For example, if the bot is driving
forward and hits a wall straight on, the input sequence might look like
this:

L 0000011111
R 0000011111

time ->

Where a 0 value means the switch is not activated and a 1 value means the
switch is activated. Each column of 0/1 values is the bumper switch value
at that point in time. We see in the above that both switches activate at
the same time.

If we hit the wall at an angle, the input might look like this:

L 0000011111
R 0000111111

In this case, the Right switch activated first, followed in 1/10 of a
second by the Left switch.

If we write code that uses only the current switch values to trigger
behaviors, there are two binary inputs each time, which means 2^2 or 4
different possible combinations of switch values. So, we can use this to
trigger a total of 4 different behaviors. The inputs would in effect have
a meaning something like this:

LR
00 Touching nothing
01 Hit something small on the right
10 Hit something small on the left
11 Hit something big.

But if we use not just the current input, but the current input and the
last input, we now have 4 binary inputs to trigger behaviors with, which
means 2^4 or 16 different combinations. And they will give us that much
more information about the state of the environment to react to. They
could mean things like this:

Prev Cur
LR LR meaning
00 00 nothing happening
00 11 Hit wall straight on
01 01 Hit something on right
01 11 Hit wall at angle
11 11 stuck.

Buy why stop at just the last two sensory input values? We can use 3, or
4, or 1000. The more historic sensory data we record, and use to trigger
our output behaviors, the more complex, and the more "intelligent" our
behaviors can become.

SO, back above, when you asked:

How many inputs can those behaviors sense?

I said infinite, because even though you have only 2 binary sensory inputs,
there's no limit on how long of a history of sensory inputs over time, you
can (in theory but not practice) record, and use as a basis for triggering
behaviors. In my simple-bot example which has 2 binary inputs sampled 10
times per second, if you were to record only 1 second of sensory data, (10
x 2 bit samples) and produce reactions to that small set of sensory data,
you would already have 2^20 or 1,048,576 different sensory conditions to
react to. If you recorded only 2 seconds of sensory data, only 40 bits,
you would have over a trillion input states to respond to.

The problem here, is that the machine needs to understand what state the
environment is in, and produce the correct reaction (behavior) for the
current state of the environment. But the current input values from the
sensors don't tell us much about the complete state of the environment. To
form a more complete picture of the state of the environment, we need to
save historic sensory data one way or another. The simple way, is to just
record a history of raw sensory data. But that doesn't work well because
we quickly end up with more data than we know what to do with. The data
must be compressed, and summarized into a smaller, and more useful, set of
information.

In most robots, the compression techniques are hand picked, and hand
crafted, by the robot designer. We might for example build a map of the
environment. How that map data is stored, and how the data is updated, is
all something the robot designers normally choose to best fit the
requirements of the problems the robot is trying to solve. But, that
internal representation of the environment (the map), can generically be
thought of as nothing more than a compressed, and summarized, history of
past sensory inputs.

The robots behaviors (its outputs) end up being triggered (aka a function
of) this internal world state, and not just the current sensory inputs.

A smart robot for example might explore the environment and find an
electrical outlet that it can plug into to recharge its batteries. And
then later, when the batteries start to run low, it can drive straight to
that outlet for a recharge. That entire behavior of driving to the outlet,
which might require making multiple turns to navigate down a hall, and into
a room, to get to the outlet it can reach, was in effect, all controlled,
by what it might have learned 2 weeks ago. Or in other words, sensory
inputs it received 2 weeks ago, is controlling the fact that it just took a
left turn now.

So, even a very simple machine, with two wheels, and very limited sensors,
still has a hard job ahead of it to act intelligent. It needs to collect,
and summarize historic sensory data, and produce different outputs, based
on that summarized data. And even a small collection of summarized sensory
data, can quickly lead to very complex behaviors.

But, as I said at the beginning. If you are trying to make a robot act
intelligent (like animals and humans), one of the main things you have to
add is learning. And that's where the problem really gets interesting.

Without learning, the robot designer will typically hand pick what type of
internal state information should be maintained, and how that sensory data
will update that state information. And they will also manually create the
algorithms for how the machine reacts to the internal state data. There is
no end to the number of algorithms that have been created like this, and
published in the literature for different robotics problems. But, to be
truly intelligent (like humans), much of that has to be replaced with
strong generic learning algorithms. The internal data structures stored by
the system, can no longer be hand selected for the needs of the job at
hand. They must use some type of generic sensory compression system. And
the behaviors produced in reaction to the internal state, must be learned,
though experience.

In the end, the machine, ends up mapping environment context (as
represented by these internal models or state information), into a
different behavior, for every possible context (aka internal machine
state). A learning machine, will not use a fixed mapping, but will evolve
the mapping over time.

The limit of the intelligence of a machine, is first and foremost limited
by the size of it's internal state - not by it's sensors or effectors. So
Helen Keller, with limited sensors (compared to normal humans), still had a
brain with the same internal state size which could still learn to produce
behaviors of complexity equal to any intelligent human.

Second, the intelligence is limited by the sophistication of the algorithms
that map sensory data to internal state changes, and which map state, to
the current outputs. Any fixed mapping is what I see as the machine's
current "knowledge" - i.e. how it currently reacts to it's understanding of
the state of the environment. This is what I think of as the machine's
current behavior set.

And third, the intelligence is limited by its learning ability - how it is
able to adjust its behavior set over time.

Most robots projects, including things like the Darpa cars, are getting
smarter by using more complex algorithms for updating internal state, and
for producing outputs based on that internal state. But they are not very
intelligent, because all this internal state has been hand-created by us
humans as we design the robot. True human intelligence, requires a generic
form of internal state, that can be adapted to all the problems human can
solve. When we write a chess program, the internal state of the
environment is handcrafted for the the need of that problem - it stores the
state of the chess board. When we build a Darpa car, the internal state is
hand crafted to the need of the car, we build a map of the environment
around the car by accumulating sensory data and we hand craft algorithms
for controlling the steering, and gas, and brake, based on the current
internal state, and the internal goals defined by the application. It's
all very hand-crafted to solve one problem.

But the brain, doesn't have hand-crafted hardware created by evolution for
playing chess, and different hand crafted hardware created by evolution for
driving a car. It uses the same, generic learning hardware handcrafted by
evolution, to do either.

To move from the smart machines we have today, which get all their "smarts"
by us hand-crafting it into them, to true intelligent machines, we must
replace all these hand-crafted algorithms that are customized for the
applications and problems we want to solve, with generic learning
algorithms, with the power to solve any of the problems humans can solve,
like playing chess, or driving a car. Not having the answer to how to
build strong generic learning into a machine, is why we currently have
smart machines, but not intelligent machines.

As a hobby, I've been looking at AI and trying to understand how to create
intelligent machines for about 30 years now. I can continue to talk more
about how I've been approaching this problem of creating strong generic
learning machines if anyone cares. Most my work has just been with
software simulations, but I've recently decided to play with robots partly
just for the fun of it, and partly, because I felt that doing more
experimentation with real world hardware might help give me some improved
insights on how to solve this problem.

So far, I've learned a lot about the current state of hobby robotics, but
haven't gotten anything done on my AI work. :)

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.

Relevant Pages

  • Re: motor emulation, modularity, feedback, prediction, and dreams
    ... Clark talks about babies learning ... in the form of pain and other sensory receptors. ... coordinate the various parts of pre-existing behaviors, ... > a programmer has access to a set of programming ...
    (comp.ai.philosophy)
  • Re: Where is behavior AI now?
    ... last input, we now have 4 binary inputs to trigger behaviors with, which ... The more historic sensory data we record, ... our output behaviors, the more complex, and the more "intelligent" our ... the robot can respond to being caught in a corner by ...
    (comp.robotics.misc)
  • Re: Temporal Learning
    ... >>had the capabilities for sensory-motor coordination, feeding behaviors, ... >>were pro or con, regards everyday survival. ... >>between ongoing activities in lower-level sensory and motor subsystems. ...
    (comp.ai.philosophy)