Re: Goal of AI: Perfect or Bounded Rationality

jalegris@xxxxxxxxxxxx wrote:

Another thing I'm not clear on is explaining classical conditioning in
terms of the operation of a reinforcement learning machine. Can you
give an example?

Classical conditioning is just the association of a behavior with a new
stimulus though pairing. Food -> salivation when paired with a bell
ringing will end up causing the bell ringing alone to trigger salivation.

However, strong reinforcement learning requires the machine to make
internal predictions about future rewards, and use those predictions as
internal reinforcers. This is what creates secondary reinforcers. This is
why money acts as a reinforcer to us even though we can't eat it, or have
sex with it, etc. Our brain has learned to recognize money as something
which leads to higher future rewards. And as such, money acts as a
reinforcer for our behavior.

In the end, a strong reinforcement learning machine must assign value to
every sensation and every behavior and use those values as reinforcers for
all behavior. Though many things have a fairly neutral value, they all must
have some value. This is done in simple computer reinforcement learning
algorithms for example when they play a board game and assign a value to
every board position. Moves in the game that lead to stronger board
positions end up being reinforced because stronger board positions are more
likely to lead to a real future reward (which might only happen when the
game is won or lost).

So, strong reinforcement learning systems are constantly learning using
internal rewards. The behavior of the machine is constantly being adjusted
based on everything that happens.

If the machine already has a behavior for salivation in response to food,
and a bell happens at the same time, that bell->salivation response is
going to be rewarded by the secondary reinforcers. And after enough
rewards, assuming some other behavior isn't being associated with the bell,
then the machine will start to produce the salivation response for the
bell.

All classical conditioning can be explained in terms of the actions of
internal rewards used by a reinforcement learning machine which is
attempting to maximize not just current rewards, but all future rewards.
In order to do that, it must be making internal predictions about potential
future rewards, and using those predictions to condition behavior instead
of using only current rewards.

The language of classical conditioning is just a simple way of explaining
the more complex effects of secondary reinforcers.

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.

Relevant Pages

  • Re: Goal of AI: Perfect or Bounded Rationality
    ... strong reinforcement learning requires the machine to make ... which leads to higher future rewards. ... a strong reinforcement learning machine must assign value to ...
    (comp.ai.philosophy)
  • Re: Mechanical Dualism versus Naturalized Epistemology
    ... > the correct path (i.e. reinforcement learning), ... > a reinforcement learning algorithm. ... >> Until you can ground your ideas in actual hardware ... > to train a multilayer network with reinforcement learning. ...
    (comp.ai.philosophy)
  • Re: Goal of AI: Perfect or Bounded Rationality
    ... Classical conditioning is just the association of a behavior with a new ... strong reinforcement learning requires the machine to make ...
    (comp.ai.philosophy)
  • Re: Goal of AI: Perfect or Bounded Rationality
    ... which will cannot be accounted for by reinforcement learning alone? ... all those and a million other things are innate behaviors produced ... (and the sensors and effectors to roughly match what humans have). ...
    (comp.ai.philosophy)
  • Re: Goal of AI: Perfect or Bounded Rationality
    ... Classical conditioning is just the association of a behavior with a new ... strong reinforcement learning requires the machine to make ... a strong reinforcement learning machine must assign value ...
    (comp.ai.philosophy)