Re: How much intelligence?

JGCASEY wrote:

If you look
at the exchange between Glen SizeMore and
Michael Olea you will see that there is a lot
of technical stuff to learn to decode what for
me was so much technobabble.

1) conditional probability distribution P(x|y)
2) stationary stochastic process
3) the capacity of a hypothesis space
4) the vc dimension of a partition space
5) the predictive information of a time series
6) the complexity of a time series
7) schedules of reinforcement

I've tried to give intuitive explanations of items 1-6 (otherwise I could
make my posts a lot more succinct). As for number 7, consider an example.
A food deprived (i.e. hungry) rat is in a chamber. Every time the rat
presses a lever a food pellet drops in a cup. That is a "fixed ratio"
schedule of reinforcement (FR) - a ratio of one "reinforcer" per
"response". One pellet per 3 presses, or one reinforcer per n repsonses,
are also FR schedules. In a "variable ratio" (VR) schedule of reinforcement
the reponses per reinforcer varies randomly about some mean (e.g. door to
door salesman, telemarketer, slot machine...).

In a "fixed interval" (FI) schedule the reinforcer is made available some
fixed amount of time after the last reinforcer was collected, but it can
only be collected as a result of a reponse. Responses before that time has
elapsed have no effect. So if the interval is 30 seconds. A food pellet,
say is ready to drop after 30 seconds. Any lever press after that causes
the pellet to drop. 30 seconds after the pellet is dropped another is ready
to drop. In a "variable interval" (VI) schedule the time between the last
pellet drop and the next pellet becomes available varies randomly about
some mean.

In a "fixed time" (FT) schedule a pellet drops at fixed time intervals
independently of what the rat does. In a "variable time" (VT) schedule a
pellet drops at time intervals that vary randomly about some mean
independently of what the rat does.

Those are some of the basic schedules. There are many variants, and also
schedules can be combined. The bigger picture is that every action you take
has consequences. Those consequences of course influence the actions you
take. Switch keyboards, or change from a 2 button mouse to a 3 button
mouse, for examlpe. Or imagine an electrician does some work and now lights
turn on when you flick the light switch down instead of up - it's been the
other way for years, and you keep flicking the switch the wrong way now and
then. Or move to a country where they drive on the left side of the road
rather than the right side. Or put on inverting goggles that turn the world
upside down. These are cases when a regular relationship between actions
and consequences makes an abrupt shift. Often in the "real world" the
relationships are not strictly regular but probabilistic. This can be
described by a conditional probability P(consequence|action), the
probability of consequences given actions. You don't make a sale every time
you give a presentation. Once you are aware of the basic idea you can see
these sorts of relationships are at work, in fact *must* be at work, in
everything we do.

There is a third element to add to actions and consequences - signals. For
example, the probable consequnces of driving through an intersection change
when a traffic light changes from green to yellow to red. The light acts as
a signal of probable consequences. It would be a good idea to heed the
signal, in which case your behavior has come under "stimulus control". I
suspect the terminology puts some people off the idea - "I'm not a passive
agent, controled by stimuli, controled by my environment!" But it's hardly
about being passive. No salesman can survive without paying attention to
environmental cues signaling potential consequences of actions. No
philosopher crosses the street when a speeding car is coming. The succinct
expression of these probabilistc relationships is
P(consequences|context,actions) - the conditional probability of
consequences for given actions under given contexts (context is my own term
for "signals" very broadly conceived - there is probably a technical term;
also "actions" is an informal term here). The extent to which behavior is
under stimulus control can be described as P(response|stimulus) the
probability over responses under given stimuli.

The terms "reinforcer" and "response" need some careful attention. The
example of a "lever press" as a response covers many distinct acts - the
rat can press the lever with the left paw, right paw, both paws, sit on it
- anything it does that results in the lever being pushed down hard enough
is the same "response". A "reinforcer" is not something assumed a priori to
act as a "reward". Food can be a reinforcer in some cases, more or less
neutral in others, and revolting (aversive) in yet others. A consequence is
a reinforcer if it results in an increased probability of responses
producing that consequence. Is this circular reasoning? Nope. It's
descriptive.

The simple "push-pull" stimulus-response notions some seem to have of what
this is all about are gross misconceptions.

So there is a crash course on some of the technobabble of the experimental
analysis of behavior. Think it has any bearing on AI?

-- Michael

.

Relevant Pages

  • Re: Challenge to the behaviourists #2
    ... key-pecking is maintained under a fixed-interval 3 min schedule ... reinforcer is delivered upon the occurrence of the first response ... progressive-ratio schedule of grain presentation in which the ...
    (sci.cognitive)
  • Re: Pigeons, People, and Priors
    ... the variance of the probability generator go to zero you have a continuum ... a random-interval 60 s schedule is not. ... The Exponential Distribution ... I probably should have used the phrase "statistical learning theory" rather ...
    (comp.ai.philosophy)
  • Re: Challenge to the behaviourists #2
    ... > reinforcer is delivered upon the occurrence of the first response ... > pigeons in terms of rates and temporal patterns of response as well as ... > progressive-ratio schedule of grain presentation in which the ...
    (sci.cognitive)
  • Re: Challenge to the behaviourists #2
    ... >>You have Oedipus, who is unknowingly in love with his mother. ... > reinforcer is delivered upon the occurrence of the first response ... > pigeons in terms of rates and temporal patterns of response as well as ... > progressive-ratio schedule of grain presentation in which the ...
    (sci.cognitive)
  • Re: Negative Slack & Multiple Critical Paths
    ... I would like the expert opinion on only ONE critical path. ... each of these tasks showing either zero slack or negative slack. ... greater than the available schedule time between your Project Start ... the probability of sucess decreases exponentially. ...
    (microsoft.public.project)