learnable dynamics (was Goal of AI...)

JGCASEY wrote:

Michael Olea wrote:
...
I tell you all this, of course, in anticipation of
ripping apart, time permitting, your latest remarks
about "The animal, not the environment". (If I get
around to it, I hope you understand it will be the
comments I am ripping, not the commentator).

Hope you get around to it then :)

Hehe.

First I want to finish a little "just for fun" code to investigate the
performance of some unsupervised learning algorithms on a time series
generated by a chaotic dynamical system. In this case the time series
itself is "complex" in the sense that it "looks random", but there are
simple rules generating the dynamics. Do the algorithms learn a compact set
of rules that are a "good" approximation of the actual rules?

The simplest chaotic dynamical system I can think of is the "logistic map":

x[i+1] = k*x[i]*(1-x[i])

This is a "discrete dynamical system", meaning we sample values at discrete
times, say every day, every month, whatever. So x could be the size of a
population of insects, which we sample every generation. The mathematical
biologist Robert May investigated the logistic map as a simplified model of
population dynamics. In this case x is not actually the number of
individuals in the population, but a fraction of some maximum possible
population size (under the assumption that the population, because of
factors like competition for finite resources and so on, can never exceed
some maximum). So in this case x can only take values ranging from 0 to 1.
So x[i] is the value of x at generation i. The equation determines how the
size of the population in one generation depends on the size of the
population in the previous generation. Here k is a constant, a "fecundity
factor" approximating the average number of surviving offspring per
individual.

Depending on the value of k and some initial population size (the first time
we observe it) there are 3 qualitative long-term outcomes:

1) the population reaches some stable size and stays there - this is a
"fixed point attractor".

2) the population oscillates between some finite number n values of x - a
"periodic attractor of period n".

3) the population is aperiodic, ("chaotic") never repeating any value it has
assumed in the past.

The main interest for population dynamics is not that this is a realistic
model, but that (seemingly random) fluctuations in population need not be
due to random outside influences, but simply reflect the inherent dynamics
of the system.

My interest is how unsupervised learning algorithms perform in the chaotic
regime, given only the stream of numbers x[i]. The "rate of growth of
excess entropy (aka predictive information" of the time series is a measure
of the "complexity" of the dynamics, which should indicate there is an
underlying simple model generating the series even though the series itself
"looks random".

Another twist is to turn the stream of values x[i] into a stream of bits
b[i] using a rule like: if (x <= 0.5) output 0 else output 1. This bit
stream has many of the properties of a sequence of tosses of a fair coin.

Anyhow, a simple C++ "functor" for the logistic map:

template<typename TREAL = float>
class LogisticMap
{
public:
//
// Constructor
//
LogisticMap(TREAL l, TREAL x_0 = 0.5)
: lambda(l), x(x_0) {}

//
// Accessors
//
TREAL Lambda() const { return lambda; }
TREAL X() const { return x; }

//
// Functor
//
TREAL operator()()
{
return x = lambda * x * (1-x);
}

private:
TREAL lambda;
TREAL x;
};

And a snippet of code that calls it:

LogisticMap<> map(4.0, 0.4);

....

for (int i = 0; i < nIterations; ++i)
{
std::cout << map() << '\n';
}

-- Michael

.