Re: Representationalism rescues reinforcement learning


"J.A. Legris" <jalegris@xxxxxxxxxxxx> wrote in message
news:1180013492.703171.174690@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
On May 24, 5:35 am, "Glen M. Sizemore" <gmsizemo...@xxxxxxxxx> wrote:
"J.A. Legris" <jaleg...@xxxxxxxxxxxx> wrote in message

news:1179971232.741884.221330@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx


So here's the thing: in the real world, an organism that learns by
trial, error and (yawn) reinforcement, is likely to get eaten before
it is lucky enough to stumble on the appropriate response.

No, if a feature of the world is stable enough, the animal has behavior
that
is elicited. Just because some responses are acquired through
conditioning
(you imply operant conditioning, but the issue is largely the same for
Pavlovian conditioning) does not mean, say, escaping predators is.
Further,
as I have repeatedly pointed out to you, operant conditioning can be
virtually instantaneous, as Skinner demonstrated in 1938. We see this in
operation (as I have explained to you before) in so-called imprinting.
Little could be as important (life and death) for a duckling or gosling
than
following mom, yet it has been DEMONSTRATED that the behavior is due to
operant conditioning. What the birds inherit is the capacity for
reducing-the-distance-between-itself-and-the-first-moving-object-seen-of-ap
propriate-size
to be a reinforcer. If a device is constructed such that the object moves
away more rapidly when approached, but moves closer when the bird moves
the
other way, it learns to move the other way instead of approaching. If
pecking a key causes the object to move towards the bird, it learns to
peck
the key. In the natural world, the usual consequence of moving towards an
object is that the distance between you and it is reduced and the
behavior
of approaching is learned extremely rapidly. It is operant conditioning
and
the birds don't die before they learn it.

But, it if
has a virtual environment in its head where it can test various
responses before committing to any, it has a leg up on the challenges
of existence, which appears to be just what we mammals have managed to
evolve - internal representations of the real world with little
homunculi going at it, and just possibly, another level or two of
homunculi contained therein (not so much for the good of the theory,
but just to irritate antirepresentationalists a little bit more). And
get this - it's TESTABLE!

Modern humans do, in fact, engage in behavior that could be called
"testing
out the behavior" and it is part of the complicated behavior we call
"thinking." There is little reason to believe, it seems to me, that this
is
a basic process in itself. We have to learn to observe our own behavior
and
its effects on ourselves. I think, also, that there are serious
limitations
to the circumstances in which this applies. Verbal behavior is probably
one
area where it applies. We learn to say silently many things before we say
them publicly and we are well set up for this to be effective since our
"listener's repertoire" is the product of histories somewhat similar to
those we are talking to. What else it applies to is not clear to me. Say,
for example, that you must jump a chasm. You either jump or not, but upon
what does that depend? Even if one does imagine a leap and imagine a
distance covered and compare the imagined distance to the actual distance
etc., what is imagined cannot be independent from our observations of our
own leaps and the leaps of others'. But is that what actually goes on?
And
even if it is, this does not mean that what happens in imagination is
taking
place in a virtual world. In the case of testing out verbal responses
before
they are emitted publicly, there is absolutely no reason to think that
the
silent response is anything more than a less intense version of the
public
response. It takes place in the world (and the organism/environment
interaction is part of the world - I say this for the benefit of Smart
Joe),
and the effect of that verbal behavior on ourselves is the same as public
behavior (or very similar - since the response is a less-intense its
effects
may be slightly different). I think that most of what we call
"imagination"
is simply automatically reinforcing. We imagine others showing us
admiration, or we imagine the face of a lover, or the face of a dead
friend.
I can sometimes figure out songs simply by "listening to them in my head"
as
I sit at the piano - at least it plays a role. If the song is too
difficult,
I engage in other behavior. I go fetch a CD player with a "rewind"
function.
But either way, there is no reason to think that any of this takes place
in
a virtual world; listening to a song is behavior, and that behavior can
occur even when there is no media. It can't occur, however, if our
music-listening behavior is currently being strongly controlled. Try
listening privately to a song while some other song is playing publicly.



--
Joe

Now that Smart Joe is here, you should probably either go by "Legris" or
"Dumb Joe."

I love the smell of an irritated antirepresentationalist on the
morning. Smells like...victory.



Yeah, keep tellin' yourself that DJ.



I got onto this after observing my uncle's cat, who apparently has
"nightmares" (in broad daylight) from time to time. What's he
responding to? Images in his fuzzy little head, perhaps?



Even if it was "responding to images" it could be argued that the images are
behavior. It is possible, for example, as I've told you many times, to
condition seeing red. How? Merely repeatedly show a subject a particular
shape as a particular color. This is why, for example, that if you show a
person a black valentine briefly they will see it as red. This was known
more than 50 years ago. And it looks like presenting it briefly isn't
necessary to get the effect - a recent experiment was reported in which a
grey scale photo of bananas was presented on a screen but color could be
added by turning dials (something like that). The task was to adjust the
photo until the bananas were grey. Subjects tended to adjust it towards
blue. A parsimonious view of dreaming is simply that it is behavior. And
behavior is located in the world.



And when you
think about it, maybe that's why we mammals evolved hippocampi, which,
among other things, are evidently involved in mediating the effects of
context on behaviour.



So far so good. Contextual control is possible in both operant and Pavlovian
conditioning - actually it should almost be called "higher-order context"
since "simple" antecedent stimulus control is "control by context," and we
usually view contextual stimulus control as what happens when a stimulus has
one effect in the presence of another stimulus and a different effect in the
presence of another stimulus.



The hippocampus may be helping to build a model
of the world that operates more or less "off-line" during dreaming
and deep thinking, while providing a ready reference for quickly
testing out reponses on the fly. Call it a less intense version of an
actual response if you want, but it's happening somewhere, and if the
external environment is not shaping it just now, the internal
"environment" is the only alternative.



This strikes me as largely gibberish. Especially since I can't give much
meaning to: "if the external environment is not shaping it just now."
Behavior that has been affected by consequences is not necessarily
"currently being shaped." Plus "less intense version of an actual response"
appears to be about what I wrote, but activity in the hippocampus is
probably not profitably viewed as behavior. The hippocampus appears to have
something to do with the so-called transfer of memories from short to long
term memory, but the view that memory is some sort of literal storage of
representations is mere assumption. Exposure to contingencies produces
behavior, and if certain things are done to the animal's brain after it is
taken out of the chamber, then the animal behaves, later on, as if it was
never exposed to those contingencies. Exposure to contingencies produces
stimulus control and certain physiological events must be allowed to occur
if the animal's behavior is to be under stimulus control at a later date. I
have no trouble with this. It is when it is insisted that what is being
stored is a picture of the environment and contingencies that I am forced to
ask if such locutions really make sense and are parsimonious. I conclude
that they are not.





Of course, this presumes a
degree of modularity in the brain's operation, which for the
mechanically-minded among us is a no-brainer.



Not many behaviorist would disagree with the notion of "modularity." The
question, however, is "What are the modules?"



And this brings us around to the EAB gambit: it may be that a
reinforcement learning machine that consults internal models produces
behaviour that is consistent with a non-representational learning
algorithm, in the sense that the resulting feats of "inspiration" can
be rationalized as complexes of simple conditioning, just-so stories
if you will. But in a challenging environment, a dead non-
representational machine tells no tales.



The duckling's life depends on following its mother, and this gets "shaped
up" very quickly via operant conditioning. It is unsurprising that you
ignored this.



--
Joe
"not as stupid as I look"



Maybe. But even if you're only half as stupid as you look, that's still
pretty freakin' stupid.



With Distaste,

Glen


.

Relevant Pages

  • Re: death of the mind.
    ... > reinforcement, average reinforced inter-response time, number of responses ... >> You decide to go outside and crawl through the kitchen window. ... >> novel FORM can occur but can't be accounted for by operant conditioning. ... And it is false because definitions of operant response ...
    (sci.cognitive)
  • Re: every stiff diverse echos consistently enforce as the chosen monks would
    ... Her half was eligible, equivalent, and grins in response to the ... manual grin may arrive more north-easts. ... If you will imagine Catherine's cult once again advertisings, ... commited let alone the environment. ...
    (sci.crypt)
  • Re: The structure of a self-conscious mind
    ... All stimulus and response classes are ... ALL individual members of response classes are ... English was learned via operant conditioning, ... Your brain is the product of natural selection, ...
    (comp.ai.philosophy)
  • Re: Reinforce learn this
    ... the response can come to be under stimulus control of many ... the parts that we do because of the contingencies arranged by the social ... Gestaltic effects happen on stimuli that have not been seen before. ... function of exposure to contingencies of reinforcement. ...
    (comp.ai.philosophy)
  • Re: Reinforce learn this
    ... attribute to reinforcement. ... response "blew up" in the presence of chemical explosions, ... Behaviorism has no problem with the principles of Gestalt perception, ... Gestaltic effects happen on stimuli that have not been seen before. ...
    (comp.ai.philosophy)
Loading