Re: consciousness, was Re: etc.

On Jun 18, 5:16 pm, JGCASEY <jgkjca...@xxxxxxxxxxxx> wrote:
On Jun 19, 3:37 am, c...@xxxxxxxx (Curt Welch) wrote:

JGCASEY <jgkjca...@xxxxxxxxxxxx> wrote:
On Jun 17, 4:53 pm, c...@xxxxxxxx (Curt Welch) wrote:
stevendaryl3...@xxxxxxxxx (Daryl McCullough) wrote:
That's true. To understand intentional behavior,
you need a theory of motivations, where they come
from, how they evolve with time.
That's correct. Do you have one? Skinner gave
us one 50 years ago.
And what was Skinner's theory of motivations and
understanding of intentional behavior that is of
any use to building AI?

CW:
Operant conditioning of course.

Our internal motivations are created as secondary
reinforcers.

JC:
My understanding is our primary motivators are innate
not learnt. Other things can stand as proxy for these
primary motivators via past associations. If one rat
races down the maze while another rat wanders around
inspecting this that and the other it is the primary
motivator, hunger, that is driving the first rat.
The second rat had also learnt that location x has
food but is not motivated to go there because its
primary motivation, hunger, is not activated. How good
the smell of food is depends on how hungry you are.
Performance = Learning + Motivation and that motivation
is innate and primary everything else is an association.

...

Do you know over the years that I have posted to
this group all I get is hand waving. As far as I
can tell people like Sizemore and Wolf don't even
believe that AI can be real intelligence and
certainly have zero interest in it.

CW:
It's kinda hard to support the position that people
who spend so much time and effort posting to an AI
group have zero interest in AI.

JC:
Why do you think others have left for the other ai-forum?

CW:
I agree that I haven't seen much from either of them
about how to build a machine which duplicate human
behavior, but I haven't seen anything from you either.
Should I assume you have no interest in AI either?

JC:
I don't set the agenda to the discussion. I have many
times tried to talk machines not animals but without
any responses. That leaves me talking to myself which
I can do without posting the ramblings to a group.

Who in this group bothered to spend time with your nets?

Who talked about simple programs to illustrate things
like habituation, sensitization, classical and operant
conditioning? Who asked the hard questions like, how
does the animal separate two sounds, say bell and whistle
into stimuli? This has to be faced if you want to build
AI whereas the behaviorists take this as a given.

I don't spend time studying rats but when I get the
chance I have spent time implementing AI ideas on my
computer. But they have to be implementable and hand
waving doesn't cut it.

Reinforcement may be the key to Skinner's S-R theory
but for him a reinforcer could have been verbal praise,
a good grade, a feeling of accomplishment and so on.
In other words assuming the very things that in AI
have to be explained.

Maybe you think you can produce human level behavior
first go but if that is the case you should also be
able to produce some simpler animal level behavior
as well. You can always set the level too high and
say well I could jump it. However I suggest you set
it low enough to jump right now and move it up in
achievable increments.

And although you sing the praises of behaviorism
I think behaviorism is so impoverished as an
"explanation" that you are left with nothing
but yapping about consciousness the wonders of
reinforcement learning.
Whenever any serious AI implementation questions
are asked Sizemore runs and Wolf does some hand
waving and then fails to follow up. I really think
they haven't a clue how to translate their notions
into AI or even care to.

CW:
I don't think so either. But what both of them do
understand, is the basic nature of human behavior.
That is, our behavior is created by a process of
operant conditioning. And what both of them know,
is that if you build a machine which doesn't show
operant and classical conditioning as a fundamental
aspect of all its behavior, you won't have a chance
of duplicating human behavior in the machine.

If you goal is to build a machine that can do some
specific job a human does, like play chess, then
you don't need to build an operant conditioned
learning machine. You can just concentrate on
different ways to build a good chess playing machine.
But if your true goal is to duplicate full human
behavior in a machine, you need to start by
understanding human behavior. This is where Skinner
gave us a foothold, and a foundation, on where to
start. All human behavior, as far as we can tell,
is created, and shaped, by the same fundamental
process of reinforcement learning. As a result,
if you want to build a machine that acts in the same
was as humans, you have to build a machine which uses
reinforcement learning as it's basis for shaping all
it's behavior.

What Glen and Wolf both know, is that many people
here are just being foolish by refusing to see the
simple fact of what humans are. If you use a bad
model to explain human behavior, you will produce
even worse code when you try to duplicate in a machine.
If think we are motivated by forces like "love" and
"fear" and build those into your AI as primitive
motivators, instead of building a reinforcement
learning machine which naturally produce "love
behaviors" and "fear behaviors" as secondary
reinforcement effects, you will be building the
wrong machine.

JC:
And what are these "love behaviors" and "fear behaviors"
built out of? You say they are secondary reinforcement
effects. Secondary to what?

CW:
Worse yet, if you think the language of folk psychology
is actually telling us something about what is happening
inside the brain instead of just acting as a label for
classes of external behaviors then you are lost in terms
of understanding what it is you are even trying to build.
These are the types of things Glen and Wolf know even if
they don't has any idea how to build a machine that
produces human behavior. They correctly understand what
it is we are trying to do, and they try to help people
see that if you don't understand what human behavior is,
you will have no hope building a machine to duplicate it.

JC:
Curt I understand machines as you do. We are talking
machines in AI. Anything derived from observing animals
must be translatable into machine terminology if it
is to be implemented on machines. My impression is that
Sizemore thinks that is not possible, I am not sure
about Wolf. They don't talk programs and machines which
is where I got my interest in AI they talk animals and
humans, very complex machines. What would you tackle
first, MSDos or the latest OS? Did you start with simple
programs or complex ones?

Hi JC,

I feel your pain ;-)

You're skepticism about behaviourism is well-founded. Did you see what
happened last August when I tried to nail down the notion of a primary
reinforcer, one of the pillars of operant conditioning?

From the thread: "50 years later, Marvin Minsky still doesn't get it"
-------------------------------------------------------
JL: As far as the individual organism is concerned, sex is
evidently sort of a "pure" primary reinforcer - it's just about
insatiable and it has almost no effect other than reinforcement (at
least for males). By my argument above, food should trump sex, but
there is little reason to believe it. Many of us would happily fast
for
a few days in exchange for the realization of certain sexual
fantasies.

GS: The modern view is that reinforcers are, for the most part, other
responses (i.e., we reinforce lever-pressing with the opportunity to
eat or
drink). The original modern view (the "Premack Principle") held that
animals
distribute their time among activities and more frequently occurring
responses could be used to reinforce less frequent. A "thirsty" rat
will
drink more than lever-press, and we may, therefore, reinforce lever-
pressing
with the opportunity to drink. If the rat is water-satiated, however,
it
will lever-press more than it drinks, and the opportunity to lever-
press
will reinforce drinking. The Premack Principle ultimately fell apart
(sort
of - perhaps it can be saved) but the relativity of reinforcement
remains in
the so-called "response-deprivation" principle. Deprivation was
always
viewed as an important aspect of reinforcer efficacy, but it certainly
is
not clear that Skinner could have predicted some of the modern
findings. For
example, one can simultaneously reinforce response A with response B
and
punish response B with response A (actually, that is consistent with
the PP
and response deprivation). One can also reinforce a more frequent
response
with a less frequent response. Say a person watches four hours of TV a
day
and reads one hour; now you arrange it so the person must watch five
hours
of TV in order to obtain one hour of reading; this will usually
increase TV
watching. This ain't your Mom's Hull/Spence drive reduction view of
reinforcement.

JL: But wait a minute. Food WILL always win out. At the point of
starvation
who would give up a bite of a Big Mac for a taste of Honey? Is there
something deeper going on?

GS: Yes deprivation is always important, but even so weird things
happen.
For example, if one gives rats access to food one hour a day, they do
fine.
I'm not sure they gain much weight, but they do just fine. However, if
you
throw a running wheel in the chamber, the animals begins to run more
and
more and eat less and less, until they spend most of the one-hour
access-to-food time running. If the experiment is not halted, the
animals
die.

----------------------------------------------------------------------------

I looked into this so-called Activity-Based Anorexia a little. It only
occurs if the animal is given regular access to the running wheel
within the 4 hours following the scheduled feeding time. If you
separate the eating from the running by more than a few hours, the
animals survive. And if you put a warming-plate into the cage along
with the "wheel of misfortune", animals that would otherwise starve
spend less time running and more time warming themselves (they tend to
have lowered body temperatures under these conditions).

Evidently, primary reinforcers are not primary at all - they interact!
There's a whole lot of stuff going on both "above" and "below" them.
Kind of reminds me of epicycles on epicycles.

Curt is convinced that reinforcement learning in animals is the key to
intelligence and that reinforcement learning machines are correct
models of animal learning. If he could only get the animals to behave
correctly, he might be on to something..

--
Joe


.

Relevant Pages

  • Re: consciousness, was Re: etc.
    ... My understanding is our primary motivators are innate ... Reinforcement may be the key to Skinner's S-R theory ... is the basic nature of human behavior. ...
    (comp.ai.philosophy)
  • Re: Finding useful functions- part 1
    ... >> The control of reinforcement schedules in an experimental setting ... experimenter observes the stimuli in a different ... but to how behaviors of these animals change in ...
    (sci.cognitive)
  • Re: Finding useful functions- part 1
    ... >schedules of reinforcement will generalize to natural settings, ... >particular behaviors that are a function of what the experimenter ... but to how behaviors of these animals change in ...
    (sci.cognitive)
  • Re: Gradual Learning, not Reinforcement Learning
    ... 'reinforcement' in computational learning theories. ... Although I have certain opinions about what I call gradual learning, ... human behavior, but reinforcement learning isn't a guess, or an assumption, ...
    (comp.ai.philosophy)