Re: Gradual Learning, not Reinforcement Learning
- From: "J.A. Legris" <jalegris@xxxxxxxxxxxx>
- Date: 17 Jul 2006 17:39:51 -0700
Glen M. Sizemore wrote:
"J.A. Legris" <jalegris@xxxxxxxxxxxx> wrote in message
news:1153167830.117066.205590@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Michael Olea wrote:
J.A. Legris wrote:
What you've described so far sounds like the Bayesian model that
Michael Olea has been describing, where an estimate of the posterior
probability of an event is updated afer each observation of the
evidence. Is this the sort of thing you have in mind? At some point,
perhaps depending on a threshold probability level, a decision would
have to be made about whether the corresponding alarm should be
triggered.
That would be where the "utility model" comes in (moving from Bayesian
Inference into Bayesian Decision Theory) - the cost and gain functions
over
consequences. So you pick the thresold to maximize expected utility. That
is, of course, a normative theory, not a descriptive one - what an agent
should do, not what particular agents do in fact do. Even so it is often
a
good model of behavior under experimental conditions. There is a
consistent
difference, I've mentioned a few times, between the normative model and a
descriptive model of "matching law" like behavior. Suppose you have two
choices A and B, and that the expected utility is 90 for A and 10 for B.
The optimal choice is pick A every time. The observed behavior is more
like
pick A 90% of the time, pick B 10% of the time. The discrepancy arises
only
if the probability distribution is known, and stationary. If the
distribution is unknown (i.e. being estimated, or "learned"), and if it
might be changing then the matching law makes more sense, has been shown
to
be optimal under some idealized conditions, and is a form of "importance
sampling", very much like particle filtering methods of approximate
Bayesian inference.
It seems like a big jump from predicting outcomes, even thousands of
them, to running interactive experiments to test the predictions. How
might that work?
That, "intervention", gets a lot of attention in Judea Pearl's second
major
book, the one on "Causality". It also has been studied in terms of "value
of information". Bayesian medical expert systems do a limited form of
this
by suggesting tests to perform in order to arrive at a diagnosis. The
role
of intervention in learning has also been studied in, for example,
developmental psychology. Discounting evidence ("let me try it, you just
aren't doing it right") is one example. It is a major theme in Allison
Gopnik's work:
http://ihd.berkeley.edu/gopnik.htm
For example:
A.Gopnik, C. Glymour, D. Sobel, L. Schulz, T. Kushnir, & D. Danks (2004).
A
theory of causal learning in children: Causal maps and Bayes nets.
Psychological Review, 111, 1, 1-31.
T. Kushnir, A. Gopnik, L Schulz, & D. Danks. (in press). Inferring hidden
causes. Proceedings of the Twenty-Fourth Annual Meeting of the Cognitive
Science Society
-- Michael
The experimental results in the first paper (starting on p.64) are
fascinating. Required reading for behaviourists! Thanks for the links.
--
Joe Legris
I looked over the paper (no, I didn't read it), and my first impression is
that this is not "must" reading for behaviorists. Or rather, it is far less
"must" reading than some of the tutorials on Bayesian analyses of coin
tosses and paper-frog jumps. But let's cut to the quick, Joe. Why do you
think it is "must" reading for behaviorists? Pitch me. After all, you can
argue that I can't be persuaded, but you know that you can get a rise out of
me.
I was referring to the first paper: A theory of causal learning in
children: Causal maps and Bayes nets. Read pages 64-71 in particular.
--
Joe Legris
.
- References:
- Gradual Learning, not Reinforcement Learning
- From: Jim Bromer
- Re: Gradual Learning, not Reinforcement Learning
- From: Glen M. Sizemore
- Re: Gradual Learning, not Reinforcement Learning
- From: Jim Bromer
- Re: Gradual Learning, not Reinforcement Learning
- From: J.A. Legris
- Re: Gradual Learning, not Reinforcement Learning
- From: Jim Bromer
- Re: Gradual Learning, not Reinforcement Learning
- From: J.A. Legris
- Re: Gradual Learning, not Reinforcement Learning
- From: Michael Olea
- Re: Gradual Learning, not Reinforcement Learning
- From: J.A. Legris
- Re: Gradual Learning, not Reinforcement Learning
- From: Glen M. Sizemore
- Gradual Learning, not Reinforcement Learning
- Prev by Date: Re: Gradual Learning, not Reinforcement Learning
- Next by Date: Re: Gradual Learning, not Reinforcement Learning
- Previous by thread: Re: Gradual Learning, not Reinforcement Learning
- Next by thread: Re: Gradual Learning, not Reinforcement Learning
- Index(es):
Relevant Pages
|
