Measuring Time to Convergence



I'm experimenting with a genetic algorithm at the moment to solve a problem.
For testing, I'm using problems that are small enough to allow a brute force attack. I then stop my genetic program when it gets with 5% of the calculated optimum fitness.


I am wanting to compare lots of different runs. Soon I'll be varying different parameters (mutation crossover, population size,etc).
So I need a way to collect statistics regarding the speed of convergence. For each setup, I run multiple models ("problems") and multiple runs with the same model.


First I tried the mean and standard deviation of the log of the number of required generations.
The reasoning behind the log was that I was seeing runs as low as a few hundred generations, and some over a million generations. With a strong positive skew in the distribution, the Normal distribution cannot be assumed.
I calculate the standard deviation to use in calculating an approximation of the sample mean's error (approximation because it isn't a normal distribution).


I'm not convinced the log solution represents the long runs correctly.

Therefore I tried without the log, and now I'm worried that they're over-represented! You need a lot of 100 generation runs to counteract a 1000000 generation run! (with a population of 90, I'm seeing a simple mean of about 3000 generations)


So what statistic is usually used to measure and compare generation counts?


-- Richard Marsden Winwaed Software Technology, http://www.winwaed.com http://www.mapping-tools.com for MapPoint tools and add-ins .