Re: About enwik and AI

Hi Mark,

You are right to "have problems with this". It rests on the nature of
"meaning". That is a deep question. There are no generally accepted
answers I'm aware of.

Personally I've been convinced for a while that the best working
definition for "meaning" is: "an organization of information". As such
I don't have a problem interpreting statistics in terms of "meaning".
It follows from my definition.

I've come across some support for this definition. (Not least what I
understand of Marcus Hutter's idea that compression is enough to
predict "intelligent" behavior.)

Here's something else I came across recently which strikes me as an
expression of basically the same ideas (in Thomas Kuhn, The Structure
of Scientific Revolutions, p.g. 44-45, citing Ludwig Wittgenstein,
Philosophical Investigations, trans. G. E. M. Anscombe, pp 31-36):

<<<
What need we know, Wittgenstein asked, in order that we apply terms
like 'chair', or 'leaf', or 'game' unequivocally and without provoking
argument?

That question is very old and has generally been answered by saying
that we must know, consciously or intuitively, what a chair, or a leaf,
or game _is_. We must, that is, grasp some set of attributes that all
games and only games have in common. Wittgenstein, however, concluded
that, given the way we use language and the sort of world to which we
apply it, there need be no such set of characteristics. Though a
discussion of _some_ of the attributes shared by a _number_ of games
or chairs or leaves often helps us learn how to employ the
corresponding term, there is no set of characteristics that is
simultaneously applicable to all members of the class and to them
alone. Instead, confronted with a previously unobserved activity, we
apply the term 'game' because what we are seeing bears a close "family
resemblance" to a number of the activities that we have previously
learned to call by that name. For Wittgenstein, in short, games, and
chairs, and leaves are natural families, each constituted by a network
of overlapping and crisscross resemblances. The existence of such a
network sufficiently accounts for our success in identifying the
corresponding object or activity.
<<<

Notice how Wittgenstein's definition of meaning is fundamentally a set.

Note also that by his definition there is no single sufficient set,
only many, mutually contradictory sets.

If you're interested in exploring such "set theoretic" ideas about
meaning I strongly recommend the whole of Kuhn's book. Of course I knew
Kuhn was famous for proposing scientific progress/knowledge was
discontinuous (and partially subjective), but I never realized how much
he had to say about the nature of knowledge itself. In fact he defines
knowledge fundamentally as sets of examples. This equivalence between
sets of examples, the original sense of "paradigm", and knowledge, is
where his famous use of the word "paradigm" in the sense of "word view"
or "scientific theory" comes from.

Note also that (I would argue) attempts to base mathematics in set
theory 100 or so years ago were not unrelated to an interpretation of
"meaning" in terms of sets.

I can give you other refs. but it might be more than would interest a
compression news group.

-Rob

markn@xxxxxxxx wrote:
Matt Mahoney wrote:
Semantics is just a statistical property of text. When we think of
words like "rain" and "snow" being semantically related, it really
means that these words are likely to appear near each other. So a
compressor seeing one word, can predict the other.

-- Matt Mahoney

I have problems with this, and while they may be rooted in ignorance,
it just sounds very perceptron-ish.

Let's say I'm in a colleagues office in one of two alterante realities.
I point to a picture on his wll and say "what's that?"

In scenario one, it's a decorative photo taken from a color enhanced
electron microscope capture. He says "a rock".

In the second, it's a google earth satellite image of some empty
landscape. He says "Iraq"

Both words are pronounced exactly the same, and I feel that what is
used to differentiate them is old-school Minsky AI semantics, where
I've built some sort of cognitive web.

I think you have to stretch too far to create a statistical model to
connect the words otherwise.

To make it fair, instead have the two sentences say:

"What's the pretty colored abstract looking photo from an electron
microscope?" "A Rock"

"What's that satellite photo?" "Iraq"

In the first case, using semantics, I'm able to disambiguate the word
without any previous statistical correlation. I'm able to jump from the
concept of an electron microscope to think about things that might be
studied in a scientific context and know that "a rock" is a pretty
reasonable answer. Even without previous statistics.

Or how about something like "My dog has shngles" (disambiguating
between the building supply and the disease. You'd have zero
statistical help on this, but semantically you'd make the unlikely jump
that dogs have diseases don't do construction.

Maybe this is all old news to everyone else and I'm in totally annoying
see-the-FAQ territory. I'm sure I would be in a different NG, but I'm
hoping that since you're trying to drag compression into some new areas
you can help with some first principles.

So anyway, to sum up, it seems to me that semantics involve a lot more
than statistics.

|
| Mark Nelson - http://marknelson.us
|

.