Re: Gradual Learning, not Reinforcement Learning

Lars wrote:
[..]
I do not think artificial rules will bring us any further, because it
restricts the system to learn something special.


However, natural systems (animals, including us) in fact have "special rules". They are the inbuilt behaviours. You seem to believe that "learning" means acquiring behaviours the system has never exhibited before. IOW, you appear to believe in some sort of tabula rasa.

What you seem to be forgetting is that all learning starts with spontaneous behaviours. These are not "random". They are produced by the animal's physiology - its neurology, its skeletal-muscular system, it's endocrine system, etc. You can't teach a pig to fly because a pig lacks the physiology needed for flying. You can teach a crow to fly to some indicated place because it can fly, period. Neither animal can be taught to sing, because their vocal apparatuses can't produce the pure tones we identify with that skill. Both can be taught to vocalise in response to cues and signals produced by their trainer or appearing in the environment - ie, both could be taught to be an excellent intruder warning system. All this is obvious, but many critics of reinforcement learning forget the obvious.

You use learning HORSE as an example, and posit HOXEL as an intermediate stage, with HORSE as an "ideal" to be achieved.

Firstly, the system must be capable of "producing language", else it cannot learn the semantic difference between HORSE and HOTEL, both of which are (relatively) easy consequences of HOXEL. But "producing language" is not at all well understood, despite many decades of intensive research (and recent confident claims of explication by such as Pinker notwithstanding.) Thus, building a system that can learn HORSE (and HOTEL, etc) requires knowledge and concepts we do not IMO as yet have.

Note that the semantic difference between HORSE and HOTEL actually appears as differences in context. That is, you have "understood" both HORSE and HOTEL when you include these words in your language responses to different combinations of cues in your environment. These combinations are so complex, and vary in so many subtle ways, that it is not at all easy to to describe them. IMO, a complete description of impossible. This difficulty sheds on light on why people respond differently to the same texts, for example. Each person discriminates different cues in the complex of cues that make up a text, each person brings additional cues, such as their past experience, to the task, and so on.

OTOH, building systems that can learn to differentiate between HORSE and HOTEL as symbol strings in text is much simpler - various statistical methods point the way, as Olea's threads illustrate. For that matter, a good spellchecker illustrates how statistics can be used for some language related tasks. But that's a long way from "learning", gradual or otherwise.

Secondly, observation of children learning language shows that it is their built-in language behaviours that are shaped to produce the specific language of their community. Babies are born with many abilities required for this. Two are: A) "babbling", ie, the ability to produce the sounds of language. B) preferential attention to language sounds when presented with sequences that include random noises. Deaf children are capable of learning a visual language. Blind people can learn to read tactile symbols. So there are clearly many other abilities (==behaviours) involved in learning a language. One of these must be the ability to string hehaviours together in sequences, and delay responses to observed (heard, read, seen, felt) sequences of cues ("symbols") until the sequence is complete -- which implies the ability to discriminate between complete and incomplete sequences. (It's at this stage, BTW, that the "marking time" neuronal firings in the cortex etc, as noted by you, are certainly involved.) Any language learning system of the kind you appear to have in mind will have to be capable of at least these behaviours.

Your notion of "gradual learning", insofar as it makes sense, seems to me a poorly understood version of reinforcement learning. It neither refutes nor replaces the notion of reinforcement.

HTH
.

Relevant Pages

  • Re: Strong AI Thesis (No Chinese room, I promise)
    ... with the idea of a generic learning system at work. ... make vision work better, and language work better, etc. ... Logic, mathematics, and scientific method ology have ... recognized the limitations which linguistic practices im pose on ...
    (comp.ai.philosophy)
  • Re: Ben G on reinforcement-learning and the wirehead problem
    ... "Reinforcement learning is learning what to do--how to map ... It is of quite unnecessary for the agent to have any opinions about, or evaluations of, any of its behaviours. ... It is merely necessary for the system's architecture to enable conditioning of behaviours. ...
    (comp.ai.philosophy)
  • The Gaelic Society of Moscow speaks out!
    ... TEN REASONS FOR LEARNING IT ... ©Gaelic Society of Moscow,2003 ... language if you have few opportunities to use it in addition, ...
    (soc.culture.scottish)
  • Re: The Gaelic Society of Moscow speaks out!
    ... TEN REASONS FOR LEARNING IT ... ©Gaelic Society of Moscow,2003 ... language if you have few opportunities to use it in addition, ...
    (soc.culture.scottish)
  • Re: Opinions on intro lisp books
    ... But Lisp is a little different, ... Some languages support one style of programming better than they ... Even if that weren't the case, I'm not sure that a language being a ... I don't believe that learning to program in CL requires more theory ...
    (comp.lang.lisp)