Re: role of language in human though process

ck <ck_NoSPAM@xxxxxxxxxxxx> wrote:
Curt Welch wrote:

Yes, its a brain function, the way an atom was just an atom until
quantum mechanics demanded other answers of us.

Quantum mechanics didn't change our understanding of the atom. It only
added to our understanding of what atoms are made up of. You talk as if
new discoveries in science, such as quantum effects, changed the things we
knew were true before hand. That's just not so. Our attempt to understand
quantum effects didn't change our our understanding of how the earth's
gravity makes an apple drop to the ground.

My point is, there
is the obvious top down view of function and then there are the
other views, one of which i suggest might start with the relationship
between energy and information.

Again, you seem to imply that when we attempt to understand quantum
effects, it's somehow changing the "top down" understand established before
hand. It doesn't. It only adds to it.

You seem to be trying to imply the higher level views are wrong. They
aren't. They are simply incomplete.

The only issue here in connection to AI is what level of physics do we have
to master in order to understand human brain behavior. Is the level of
F=MA that we understand at the level of apples dropping to the ground low
enough? Or do we have to master quantum mechanics and something new we
haven't yet reached below that level in physics before we understand what
the brain is doing and how it's doing?

Since no one has built a machine which duplicates human brain function and
human behavior anywhere near close enough yet, we simply don't know for
sure what level of understanding is actually needed.

However, of what we currently understand, there's really no strong
indication that we need to understand physics better in order to solve this
AI problem. When you go around speculating about unknown connections
between information and energy, you are strongly implying that there is
something in physics we don't yet understand which we will have to
understand before we can solve this AI problem of building machines that
duplicate human level behavior. I don't buy that view at all. I believe
that physics is already miles ahead of what we need to understand to solve
AI. I believe that the only knowledge missing is improved algorithms and
machine designs for reinforcement learning machines.

Many people that debate AI seem to simply not know what the answer as.
They just don't have a clue what to do in order to solve AI and they don't
have a clue why people haven't already solved it. So, they look for
answers to explain why no knows the answer. They tend to be naturally
attracted to the idea that since they don't understand the problem, and no
one else has been able to solve it, there must be some fundamental class of
information still missing. So they naturally seem to gravitate to any area
of science, or area of their imagination, that implies there is something
we don't yet understand.

The classic and most common idea turned to is quantum mechanics. Since our
ability to explain causality vanishes at the level of quantum mechanics,
the missing answer to all hard problems, (so the argument seems to go) must
be hidden in there somewhere. It's just hogwash to me because I understand
what the problem is and it's not a mystery to me so I don't feel the need
to look under every potential rug to find the missing answer. There is
only one missing answer and far too few people seem to be able to grasp
this fact and far too few people spend any time working on. It's just a
reinforcement learning problem.

How do you take 10 million or so real time sensory inputs, and map them to
10 thousand or so real time outputs, and make the system adapt the mapping
in real time in order to solve human type behavior production problems.
That's it. That's the AI problem. It's a reinforcement learning algorithm
of the class which no one has yet built a machine to solve, but which we
understand in fairly good detail exactly what the problem is. And there is
no odd mystery of things like what is "information", or what is "qualia" or
what is "thought", or what is "feelings". All those problems have been
answered. The only problem not answered, is how to build a reinforcement
learning machine to deal with problems on the scale that brain deals with.

As for making the problem much harder
than the evidence available to us, this is precisely my point. If
the problem were as simply as the evidence available to us the
problem would already be solved.

The problem isn't simple. That's why it hasn't been solved. Not because
the problem isn't understood or is hidden under some rock no yet turned
over. Build for me, a machine that can take 100 million real time sensory
inputs with the rough bandwidth of each of the 100 million or so sensory
inputs the brain can process (maybe hundreds of bits per second per input)
and produce thousands of outputs with similar bandwidth and make all the
mapping adjustable through reinforcement learning. No one has produced a
reinforcement learning algorithm that can do this. When they do, they will
have solved AI.

For example, just build a two legged robot with a few high-bandwidth
sensors (like video and other standard sensors) with a large number of
degrees of freedom in it's legs and body, and have it learn to walk on two
legs on it's own through reinforcement learning. As far as I know, this
has never been done - but it's exactly the problem that must be solved to
solve AI. Learning systems have been applied to making robots walk, but
the only ones I know of are very limited and very specialized and couldn't
for example use arbitrate high bandwidth sensory inputs like video data to
drive it. But until we know how to do that, we won't have solved AI.

Solving these problems, which is what needs to be solved to solve AI,
doesn't require us to understand anything new about energy, or information.
Which is why I think you are way off base when you start to wonder if
there's a connection there that needs to be better understood before we can
solve AI.

There is surely room for other
theories here, other ways to look at that evidence.

Until the answer is found, there will be room for as many theories as
people can generate. I just happen to believe most of them are way off
base when there's no need for them to be since some of us do actually
understand what the real problem here is.

It's a reinforcement learning machine. That's where value comes from.
If you don't understand how that is true, read a good book or two on
reinforcement learning.

Reinforcement learning machine seems again a top down view of the
machine. Its what we see. My question is what drives its core
functions? What does each function see of the stimuli presented
to it. This other view says in effect, our various functions are
in competition for energy and resources.

It's true that the simple view of reinforcement learning is a macro level
description. To build it, we must fracture the problem down to simpler
components we can actually build. Understanding how to perform that
fracturing is the key to solving AI. My current best approach is to
fracture the high level problem down to a low level pulse sorting problem.
At the lowest level, the smallest component of the machine is simply making
binary pulse sorting decisions, and those binary pulse sorting decisions
are trained by reinforcement learning.

You however, seem to be suggesting a fracturing, which is far more complex.
And you are suggesting that there's a competition for energy and resources
between these components. I don't happen to believe that's a useful way to
fracture the problem. That's a very valid description of the problem which
evolution is solving - but it's not the problem which our reinforcement
learning brain is solving. Our big toe is not competing with our wrist
bone for energy and resources. At least I don't believe it's very useful
to look at their function in those terms. And I don't think it's useful to
look at the brain, or it's sub-parts in those terms.

Reinforcement learning machines are motivated by the part of the hardware
called the "critic" in reinforcement learning terminology. It's the design
of the critic which creates the prime motivations of the learning system.
When you build a robot for example, that uses reinforcement learning, you
must also build the critic, to define what it's prime motivations are. The
critic is that part of the hardware which maps potentially many sensory
signals, using any complex function desired, into a single dimension reward
signal which defines the machines core values. In a robot, we could for
example, give it a critic which generates a reward signal in response to
light. Such a robot would learn to be a light-seeking robot. It would
however, have ZERO survival motivations. It would drive itself right over a
cliff if it thought that would give it more light. It would have zero
understand of energy and conservation - and the only resource it would
value, is light - or the resources which helped it get more light.

We could change the design of our robot and give it a critic which
generated rewards for darkness. That change would make it learn light
avoidance behavior instead of light seeking behavior. This new version of
the robot would seek out resources which were valuable in avoiding light
(like shade trees).

These two robots would develop completely different sets of morals, and
values, because one would love light, and fear darkness, and the other
would love darkness, and fear light. But yet, both could use the exact
same reinforcement learning algorithm. The only difference, would be the
critic hardware.

Humans have critic hardware that creates in us, values that happen to be
well aligned with survival. Energy happens to be a key resource for
survival, and as such, energy is something of high value to us. But our
need for energy is not built into the design of our reinforcement learning
systems, it's built into the part of the brain which acts as the critic -
the parts which determine what stimulus signals act as rewards, and what
acts as a punishment. Food acts as a reward, because food (aka energy in
the form we can most easily use) is a very valuable resource for survival.
If we run out of energy (aka food) our odds of survival go way down.

Building the critic is actually the easy part of building a human level AI.
WE basically already know how to do that well enough to consider that to no
longer be the problem. What we don't know how to do, is build a real time
reinforcement learning system that can deal with the type, and amount, of
data the brain deals so well with.

The reinforcement learning algorithm which we have to figure out how to
build, isn't the part that makes the robot energy seeking, or energy
avoiding. So the whole issue of the importance of energy (which is key to
human behavior - aka survival), really isn't part of the problem we need to
solve at all. That part has already been figured out. You make the critic
generate rewards for behaviors which are energy saving or energy collecting
(like battery charging - aka the robot version of eating) and there you
have a robot with basic survival needs as a core value.

#
Function X has no knowledge of function Y, instead it pursues its
function, or its evolved definition. It looks selfishly for
whatever might support its goals, or provide its rewards. In this
case energy, or as you say, the satisfaction of its chemical
potential. Who did this come to be, how is this balanced? This
competition with the next function, is only tempered by the 'harm'
which might eventually result to itself.

Pick up any book on reinforcement learning (the computer science field -
not the field of psychology) and you fill find a lot of answers to how
reinforcement learning algorithms (the ones we already know all about)
balance needs. It's a simple and well understood problem of mathematics -
the problem of how the value function in reinforcement learning systems is
calculated.

As for Value, this is found in a complex relationship of 'interests'.

Yes, it's the core problem of reinforcement learning. Have you read read
anything about reinforcement learning algorithms? Calculating a complex
value function is what they all do. If you have N choices to make about
which micro behavior to create next, how does a reinforcement learning
system make the choice of what to do next? It does it by creating a value
function which assigns a value to each possible action and it picks actions
with higher expected values. The calculation of these values is what
reinforcement learning is all about.

Its not one fixed simply learned quantity.

Very true. Reinforcement learning algorithms create extremely complex
value functions which are able to calculate the value of any behavior, in
any context, which the machine is cablecable of.

The critic hardware generates a current reward signal, but the
reinforcement learning algorithm calculates an expected long term return -
aka it calculates a value which is the best estimate it can calculate for
future rewards. Aka, if the robot raises it's arm now, instead of lowering
it, will it produce more future rewards, or less? This is the problem the
reinforcement learning system is trying to calculate - the better it is at
accurately predicting future rewards, the more "intelligent" its behavior
becomes.

Building a machine which is as good as a human brain at calculating future
rewards, is what no one has yet done - but when they do it - they will have
solved AI (aka human level behavior from a machine). But, instead of
working on better reinforcement learning algorithms for better predicting
future rewards, you want to explore the idea that there's some missing
understanding about the link between information and energy. Are you
beginning to grasp why I think you are asking the wrong questions?

My question is how would
a machine appreciate this relationship? How would one connect the
machine to this idea of value, so that it meant something to the
machine?

All current reinforcement learning systems have already answered that
question. It might be a mystery to you, but it's a problem that for me,
was solved long ago. All reinforcement learning algorithms have answered
the question of how value means something to the machine.

You will no doubt murmur reinforcement learning,

Actually, I yell it at the top of my lungs. :)

but what would you be reinforcing? A preponderance of sought
associations, it seems to me, would miss this question of intrinsic
value.

You reinforce the mapping of context to behavior. Or in other words, you
calculate a value for all possible behaviors in all possible contexts.
When the machine is rewarded, it adds that data to its' accumulated
knowledge about past rewards in past contexts and uses that make the best
prediction possible about expected long term future rewards.

All human concepts of value are explained in this simple problem of picking
behavior based on estimates of long term rewards. Do we turn right, at
this point in the road or do we turn left? Which one will produce the
highest expected future rewards for the given context? That's the problem
the machine must solve. Understanding how the machine can define "current
context" is a huge part of the practical problem of how to build such a
machine. It's easy to understand in context, but very hard to figure out a
good way to build it for human level problems.

IMO we've exhausted the idea that *intelligence* is just an
information processing problem.

IMO, that's about as false as you can get. Not only do we know it's
just an information processing problem, we know exactly what type of
information processing problem it is - a reinforcement learning
problem. The only thing we don't know is how to build reinforcement
learning machines which equal the power of the human brain.

I guess this satisfies as THE answer, until the next theory comes along.

That's always possible. But the idea that we need to build a machine which
is a reinforcement learning machine is supported not just as theory, but by
the simple testable facts of human behavior. Human behavior is controlled
by conditioning. We can make people do just about anything we want them to
do by conditioning them in a carefully controlled environment. We can
force them to pick the yellow chair more than the red chair and they won't
even know we are doing it to them, but they will do it - every time. The
fact that human behavior is regulated and controlled by conditioning isn't
just on current theory that is likely to change in the future. It's a
fact. If the goal in AI is to produce a machine which acts like a human,
it damn well has to be a machine that changes it's behavior based on
conditioning or else it won't be "human behavior".

What most people will do in response to this argument is argue that even if
some behavior can be conditioned, that our full behavior is more than just
conditioning - that maybe lower animals are mostly just conditioned but
that humans have far more complex systems at work regulating our behavior.
But yet, everyone I've seen try to make that argument clearly shows a huge
lack in understanding the power of conditioning.

Again the top down view. Its just what we now understand it to be,
the latest technology applied to biology, seen in terms of the tools
available to us.

This is true. And I don't have any issue with your idea that we will
develop new tools in the future and that those new tools could give us an
understanding which we don't currently have.

However, reinforcement learning explains basically everything humans do -
from language, to thought, to planing, to feelings, to desires, to pain, to
pleasure, on and on. The only people I've seen who argue against
reinforcement being the key problem of AI are the ones who clearly don't
understand reinforcement learning and it's power. They think it's too
simple, because they don't understand its complexity.

The reason I take the time to write the long ranting messages every now and
again, is to try and wake a few people up to the idea that we don't need to
go fishing for new theories in what may look like hidden places like the
relationship between energy and information. Reinforcement learning
already explains all human behavior. What we don't know, is how to build a
reinforcement learning machine with the same power the human brain has.
Reinforcement learning theory is the only theory we currently need to
explain human level behavior. It's just a well framed engineering problem
now. Not a physics problem or a philosophy problem.

What is it of its self, what is it at the cellular
level. What is it against an evolutionary scale. Think community,
think evolved organization, think selfish function divorced from
the objective we provide the whole, what is it at its most basic
levels?

A distributed reinforcement learning system which works something like my
pulse sorting networks which at the most basic level, is nothing but a huge
network making binary pulse sorting decisions. This is the type of
solution which I believe will ultimately lead us to building machines that
duplicate human behavior.

A hundred years ago we thought in terms of engines, before that in
terms of clock work automatas, these days its information processing.
What else might it be?

Sure, there could be "something else". But I see no need for it.
Information processing is all we need to understand to solve AI in my
belief. We don't need more physics, we just need some more information
processing machine designs.

The problem I have is that most people still see many issues of AI unsolved
- not just the big one, but all these little philosophy problems as well,
like what is pain, or what is information, or what is awareness, or
consciousness? But I don't. All those questions have already been
answered. If you think those sorts of questions haven't been answered,
then it leaves you believing something big is still missing from our
understanding and that ideas of information processing alone will never
answer the missing big problems. I just can't agree with that because I
don't' think there are any big questions unanswered. All the big questions
are answered by understanding that human behavior is nothing more than the
result of reinforcement learning machine interacting with a complex
behavior. The details to work out (the questions still unanswered) are
only the details of building better reinforcement learning systems.

There is no confusion, instead there is a question. What is information
if its not about energy? what would information, and thus intelligence,
be, if it were defined in terms of energy?

It's been defined. It's a reinforcement learning machine. That alone,
answers all questions about what is information and what is intelligence.
If you don't understand how that is the answer, then you don't understand
reinforcement learning - which is why I'm writing this. If you want to
understand the answer, study reinforcement learning systems, and figure out
for yourself how it is the answer to all those questions.

I understand what we accept, i am asking what else is there?

I am asking why you think there needs to be something else when the answers
we have already answer all the questions?

To explain how apples fall, all you need is an understanding of the concept
of gravity and the understanding of some fairly simple math which is the
language we use to define what we mean by "gravity".

If you understand why apples fall to this level, why is that you think you
need more to explain why apples fall? If someone would suggest to you that
maybe the key to understanding apples falling is in the relation between
their shape and their color how would you answer them? Would you say we
already understand why apples fall and it has nothing to do with their
color?

This is basically _exactly_ how I feel about what you are asking. I
understand what human behavior is and what intelligence as well as I
understand what apple falling behavior is. Human behavior is just the
behavior of a reinforcement learning machine. There really is no mystery
to it.

The people that think AI is a mystery are the ones who fail to understand
what a reinforcement learning machine is and/or fail to understand how all
these language terms we use to describe human behavior is just us
describing the behavior of a large and advanced reinforcement learning
machine.

What else
might there be. This is an inquiry with an open mind, I am not merely
looking to exchange what we currently believe we know, call it a
pursuit for new thought.

And I'm asking why you are looking for new thought when we already have the
answer?

I know exactly why more progress hasn't been made in AI over the past 50
years. It's because most people are in the same boat you are in. They
don't understand that AI is just a reinforcement learning problem and
because they have failed to understand that, they have been looking for
answers anywhere else they could dream up to look for an answer. How can
you expect progress to be made in AI when most people working on AI are
working on the wrong problem?

The real problem of AI, is that people think they know things about what it
is to be human because they are a human, and this blinds them to being able
to see the simple truth about what humans are. Why has behaviorism been so
completely rejected in AI and in philosophy? Because people couldn't
accept and believe that they were just reinforcement learning machines
(that's would make us too stupid) - so they rejected the answers that
behaviorism gave us 50 years ago, and started looking elsewhere for the
answer. And guess what, they have made very little progress in 50 years
because they have been working on the wrong problem. Instead spending that
time to develop out better reinforcement learning machines, they rejected
reinforcement learning as being too simple of an answer, and they went
looking for something more complex.

You can't find the answer if you look in the wrong place. My constant
ranting about reinforcement learning here has just been an attempt to get
more people to start looking in the right place and stop wasting their time
looking at odd and totally unimportant places like the "connection between
information and energy".

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.

Relevant Pages

  • Re: Classical wave theory and tired light
    ... Note that the understanding of a theory that successfully models ... Straight line motion of photons involves no ... possible loss of energy. ... with respect to the deflecting massive bodies that the relative ...
    (sci.physics)
  • Re: Article: Researchers may have solved information loss paradox to find black holes do not form
    ... that arose due to heat created a very cold region in this energy ... to perceive the greatest of objects with a sense of understanding, ... windcupters moving from street to flying in the sky, ... Morning awakening, Bush is on his knees, receiving the honor, then ...
    (sci.astro)
  • Re: Classical wave theory and tired light
    ... Note that the understanding of a theory that successfully models ... Straight line motion of photons involves no ... possible loss of energy. ... with respect to the deflecting massive bodies that the relative ...
    (sci.physics)
  • [News][Comment] The bluffers guide to life, the universe and everything.
    ... he explains why a few key scientific concepts can change your life. ... understanding of how the universe came into being, how it's expanding, ... BIG IDEA NUMBER 4: DNA ... know that if a photon doesn't have enough energy (and the radio waves ...
    (talk.origins)
Loading