Re: Towards a theory of the semiotic mind-body link

casey <jgkjcasey@xxxxxxxxxxxx> wrote:
On May 7, 12:51=A0pm, c...@xxxxxxxx (Curt Welch) wrote:

But does that mean that 0100 and 1100 should be given
the same value in terms of winning a game? In backgammon
this is the case. In chess it is often not the case.
Similarity in an ANN used as an evaluator is similar in
that way. Let's consider some crude chess heuristics.
We start with giving a piece a value. We may use those
values to calculate a piece advantage.

advantage = queen*9+rook*5+bishop*3+knight*3+pawn

There is also position advantage which come under
measurements of attack, defence of king, development
of pieces, control of center, mobility, pawn structure
and so on ...

Now how would all that and more come out of a simple
network?

By clustering states to form abstractions based on similarity in the
results they create. It's the same way we form these abstractions.

It works by first automatically factoring the board state into lots of
small sub-states, and then clustering those based on temporal associations
- that is, how close in time they tend to show up in the environment.
Those associations can then be further refined based on the rewards they
produce - aka the odds of such a state predicting a win or a loss. If one
state is a strong predictor of a win, and another state is a strong
predictor of a loss, then their association (closeness in the clustering
system) is reduced.

The trick of course, if we see some state of the board as "good" or "bad",
such as "control of the center", then the net can not learn the value of
such a state, until it first "sees" the state. That is, it must first
build an internal representation of that state of the board before it can
learn it's value.

But how might "control of the center" be recognized? I'm not much of a
chess player so coming with good examples is hard, but I'll try to wing it
to see if I can create a general idea.

In the starting position, the queen can not attack any of the center
squares becuase she is blocked by a pawn. But if the squares in front of
the queen are empty, then she is attacking the center. So if the network
can simply recognize the sensory pattern of the queen being in her starting
square, and the 4 squares in front of the queen being empty, that pattern
on the chess board represents one small part of putting pressure on the
center of the board.

So if the general "sensory input" to the system is the state of the
environment, that is, the board state, a network would need to create
internal signals that represent various state patterns that were common on
the board (such as queen at d1 with d2 and d3 empty).

How the network would select what sub-states to represent would depend on
what states it happened to see in the games it played. It would adjust the
definitions of the sub states so they were roughly equal in probability
over time. And it would create associations between the states based on
the order they tended to show up in real games.

rewards (aka game wins and losses) would be back propagated though states
based on their temporal associations. As certain states become obvious
"bad" states, and other states become obvious "good" states, the program
would tend to favor the good states and avoid the bad states which in turn,
changes the association between those states, and in turn, causes the "Good
states" to be clustered more strongly together.

In other words, before the network learns much of anything about what is
good, it will choose all moves with nearly equal probability. That causes
the network to configure itself to represent all possible board positions
reached by random play roughly equally. But as it starts to learn basic
ideas that some of those states are worse than others, it will start
selecting those states, so those "bad states" will be seen less often in
the typical games the network plays. Because they are seen less often, the
way the network has configured itself to represent sub-states will change.
Less state information is allocated to the "bad" states, and more is
allocated to the good states. This causes the network to gain a high
resolution representation for the good sates. That is more network
resources are allocated to represent good states because in time, only the
good states are the ones the network "sees" all the times.

As more network resources are allocated to represent the good states, that
also allows it to create more complex understanding of the good state,
which, if we were lucky, would allow it to form a more complex defined
pattern that could represent something like "control over the center" by
combing together lots of simpler sub-states like "queen attacks center",
AND "rook attacks center", etc.

So the network forms abstractions by recognizing common sub-patterns in the
sensory environment (like a a visual system learns to recognize the common
sub-pattern of a vertical edge), the system learns the value of these sub
patterns based on their prediction of a win or a loss (just by playing lots
of games), play improves slightly by avoiding the really stupid moves,
which then allows the network to re-define the sub-patterns it recognized
based on what it's now improved play is seeing more of, which gives it a
"better understanding" of the "good positions" it sees a lot more of now,
which then allows it to recognize more complex "good" positions, which then
allows it to play a little smarter, etc.

So the network abstraction system seeks out the abstractions (sensory
sub-patterns) that are most useful to wining chess games.

This I'm sure is not very clear, but that's the sort of high level idea of
how a network will learn to create the right heuristics, such as "control
the center". But for such a small network, the heuristic would just be a
strong gut feeling that some complex board position was "better" than some
other and though we might use language to describe it as "control the
center" this simpler system (one that hasn't learned to regulate it's chess
playing by talking to itself), would simply "see" those board positions as
"better".

The backgammon ANN gives a value to some game
state based on a win or a loss in previous games.
A similar state (as defined above) is likely to give
a win/loss as determined by other similar states.

How is 1101 a "backgammon board state"?

In chess two similar states (as defined above) may
generate completely opposite win/loss outcomes.

You have totally failed to make your point about what a similar state in
backgammon is, and what a similar state in chess is and how one is so
"obvious" and the other not.

I haven't played backgammon in so long I don't really even remember how to
play. But I believe there's a rule where if you have two or more pieces on
the same point, you control that point. If you have only one piece, then
the other player can land on that position, which puts your piece back to
the start, which sets you back in the game. As such, a board position
where you one piece on a point is very different value than a board
position with two pieces on the same point. So imagine two board positions
where the only difference is that one piece is moved from where it's a
double on the same point, to the point next to it, so that two points have
only a signle piece on it.

These two board positions are very similar, yet they have extremely
different strategic value. One is to be avoided at all costs, the other
might be considered a strong position. How is it obvious (as you try to
suggest) to the network that these to positions are not in fact "close in
value" as you suggest?

Some board positions are strong (good), and some which are very similar are
weak (very bad). TD-Gammon has some power to learn this, and to adjust
it's network associations to reflect this fact. Backgammon is not "easy"
because "similar board positions have similar strength" because similar
board positions don't always have similar values. I don't know backgammon
at all, but I know it well enough to know what you are suggesting is just
dead wrong.

It might very well be true that chess positions have much greater variation
between similar positions than backgammon, meaning a similar approach for
chess would require the network to learn more complex abstractions to play
as well. I can agree with that. But the general approach that works for
TD-Gammon should work for chess, IF (and that's the big IF), the network as
designed has the power to form the needed abstractions (state patterns) to
correctly represent the important aspects of the state (such as the example
of control the center). If the network can't be configured to represent
some measure of "control of the center", then it can't hope to ever learn
to do it. So the key to making such an approach work for a game of chess,
is to have a network that can configure itself to recognize these important
patterns.

It's the missing algorithm I keep talking about over
and over in these posts.

And I have suggested it may not be a missing algorithm
so much as missing algorithms. There is a difference
between a built in algorithm and one generated by an
algorithm. Evolution built innate algorithms as hardware
and I doubt that the human brain doesn't utilize them
even if it relies this general purpose learning system
which itself I believe evolved out of a not so general
purpose learning system.

I could be wrong but I feel it is more probable that
evolution kept on doing what it has always done when
it evolved the human brain. That learned high level
behaviors have been implemented by using innate high
level behavior mechanisms such as a cat burying its
poo. Where is that behaviour stored in the dna?

If you understand the true nature of the learning problem, you wouldn't
cling so hard to your belief that there are lots of different hard wired
modules "helping out".

Hard wire modules can't make _generic_ learning easier, they can only get
in the way. They only work to make limited domain learning easier. You
can make some argument for hard wired modules to help out with any part of
the environment which has been fairly constant for a very long time
(millions of years). The way the eyes work and create a 2D view of a 3D
environment is something that fits the bill as "constant part of the
environment for millions of years", and as such, could have associated
hard-wired hardware to help process that sensory data.

But once you get past that, and even before you get past it, we run into
the problem that generic learning has to have access to these low levels.
At some point in time, "red stuff" might have been very good to humans
(bloody food to eat). But a thousand years later, for humans living in a
farm environment, "red stuff like blood" might have been an strong
indicator of "bad" such as humans fighting and killing each other. Or a
human living in the far north, that has to spend most his life looking at
snow, has need to develop a visual decoding system based on what he seems
the most of - different patterns of white, alone with a sharp eye for
detecting little dark dots (seals) way off in the distance. Where as
someone who was raised in the jungle, see very little white stuff, but lots
of shakes of green - and almost nothing "far way" becuase they spend so
much time in dense jungle they seldom see anything more than 30 ft away
from them. If the generic learning starts at the lowest levels of the
visual system, then the visual system can 1) adapt to the type of sensory
patterns it sees in real life, instead of being hard coded to expect only
one distribution of typical patterns (which is something generic learning
must do at the higher levels), and 2), it can make reward associations with
these same patterns to guide it's actions (which is what generic learning
MUST do at the high levels).

If you understood better the problem we must solve at the high level to
support strong generic learning so we can learn all these complex behaviors
and complex ability to recognize patterns that only recently showed up in
our environment (like recognizing the visual pattern of a chess board where
our pieces are controlling the center), you would understand that wasting
time trying to hard-code low level systems that must also use these same
generic sensory pattern processing algorithms, is not very productive.

It makes no difference how much is, or is not, strong generic learning in
the brain. The brain is built like it is for 1) complex historic reasons
that are a function of it's evolutionary design path, and 2) what has
worked well for keeping humans alive for the past millions of years.

Neither of those two things are important to AI. We aren't building our AI
machines out of neurons, so brain hardware design constraints created by an
odd history of how evolution configured nuerons, and made them self-wire by
growth into a brain, is not important stuff for us who are building AI out
of software.

And likwise, until we need or want to try and duplicate all the subtle
surival skills and instincts of humans, we can ignore all that as well
becuase we are only interested in creating human intelligence first. We
can work on an AI with a human personality and fears and limitations later.
And to create human intelligence, we need that strong generic learning that
can allow the same generic machine to learn to use it's hands and arms to
play chess to get rewards, and then use those same hands and arms to cook a
meal to get a reward. And then use those same arms and hands to type of a
computer to get rewards. These behaviors, and all the other millions of
things we can learn to do with are arms and hands that evolution never saw
until 100 years ago, and as such hasn't provided us with any specialized
hardware (other than our hands and arms, and generic learning control
system called the brain), has to be solved with strong generic learning.

So now, you move the goal post further. Not only does
it have to create a unique design for an antenna, it
also has to make the DECISION, to do that instead of
watch TV before it can be considered creative?

Does it also have to learn English, and French, and
Spanish, and write a few books on Antenna design and
get itself hired as an EE at NASA before we can say
the machine has created something new?

I never said the unique design was not a creative act.
Build a machine to generate art and it will produce
many creative acts. Creativity is over rated and in
humans is usually the result of a lot of trials in
a subject of immense interest. It only looks like an
act of creation out of nothing, when it fact is was
a simple search procedure. Nothing more amazing than
any search algorithm with a way of determining that
something being found that fits some definition of
success.

Yes, but I wrote about it because the OP said he didn't think machines
could be creative when they already are creative. And then you chimed in
and said something to the effect that my examples were not examples of
creativity.

That would be the 1 thing I want computers doing, to
regard them as "thinking"

They are already "thinking".


Word magic based on a limited understanding of what a
human brain is doing when it is "thinking".

That's word un-magic based on the fact that the brain
isn't doing anything special when it's "thinking" other
than firing a few neurons.

That is as silly as saying that a random collection
of logic gates is doing arithmetic. Doing arithmetic
IS special. It is not something any old collection
of logic gates can do. Thinking involves the "firing
of neurons" and non thinking is also involves the
"firing of neurons" but they are not the same activity.

Well, I would bother to listen to you if you had even ONE suggesting as
what thinking was, but since your normal approach in these debates is to
answer with "I don't know what it is, but it's not what you say it is",
this debate with you is pointless.

Whether you like my suggestions or not, I do actually put my neck out on
the line and tell you what I think these hings are in real mechanical
terms. And the only response you come back with, is "no, you are wrong,
even though I don't have a suggestion as to what is right".

How many transistors does the computer have to fire
before you will consider it to be "thinking"?

It has nothing to do with how many neurons are firing,
it is what the firing is amounting to. A person having
an epileptic fit isn't thinking but their neurons are
sure firing away.

Hard to say they aren't thinking really since the term is really not
defined.

What I fear here, is that you believe thinking means "that odd feeling of
being aware of stuff that we associate with consciousness". And since you
argue that "consciousness" is some complex process we don't yet understand
and don't yet know how to build, I expect you to drag "thinking" into that
same logical argument as "something no one understands yet and that no one
has built yet".

Without us they are nothing.

Without water, we are dead meat. So what does that
have to do with it? Our dependency on water doesn't
mean when we design an antenna it wasn't our work.

And I am not suggesting it wasn't the work of the
program. I simply said it didn't decide to design an
antenna which is part of what we call thinking.

Sure, but I didn't write any of that to prove an points about thinking. I
was answer the question about creativity.

The point here is that just because we are the reason
the software and computer exists, doesn't mean the
computer isn't doing something creative.

I never said it wasn't creative.

We define them and set their parameters.


Evolution designed us and set our parameters. And
now, like or not, we are these funny little survival
machines that devote so much time and energy to
playing games that help us survive. We are only
here because evolution created us. Does that mean
it should get all the credit for us being here and
not us?

The word "credit" usually implies that the person had
a choice. Giving credit is a social activity as a form
of interpersonal control.

That's right. Where as TD-Gammon only "thinks about"
moves in a backgammon game, we use our big brains to
think about a million other things as well. But is the
"thinking" process really that different, or is the
only significant difference what we are able and do
think about?

I don't believe TD-Gammon thinks about moves in
backgammon or anything else for that matter anymore
than a rock rolling down a hill thinks about where
it is going.

... you should stop playing with words,

The pot calling the kettle black.

I build and talk about hardware,

And play with words to imply you have done more than
you have really done imho.

By the way have you ever programmed a working ANN
like the one used in TD-Gammon?

Yes.

JC

--
Curt Welch http://CurtWelch.Com/
curt@xxxxxxxx http://NewsReader.Com/
.

Relevant Pages

  • Re: DirectPlay troubleshooting
    ... settings in your Contro Panel / Network Connections.. ... MS Games Help and Support Center: http://support.microsoft.com/?pr=gms ... | that when I try to set up a DirectPlay chat session between ... For more in depth troubleshooting, try this MS Network Troubleshooting article: ...
    (microsoft.public.games)
  • Re: Rainbow Six Vegas 2 (PS3) plagued by online problems
    ... I assume happens on the PS3. ... wireless network than console owners. ... excuse to over charge. ... I have more 360 games than I do PS3 ...
    (alt.games.video.xbox)
  • Re: MCE 2005 Xbox 360 Extender PC Setup Cannot find Xbox 360 on Ne
    ... the only Network Service I have enabled is the "Internet Gateway ... Gateway Device Discovery and Control Client), the Internet Gateway Device ...
    (microsoft.public.windows.mediacenter)
  • Re: what mess is this?
    ... how we can get it configured so you can stay in the games ... Make certain that your Network adapter or Internal Modem ... HOW TO IMPROVE YOUR CONNECTION: ... | reason of those constant boots... ...
    (microsoft.public.windowsxp.games)
  • OLN wants to challenge ESPN for sports dominance
    ... No. 1 cable company may bid for NHL, NFL games to challenge top sports ... may be weighing a sports network to challenge the leadership ... of ESPN, according to a published report. ...
    (rec.arts.tv)