Re: Hutter prize entries?

A very tough situation. I find it interesting how something which we
would normally
toss up to legal nonsense is now something which we all wish was worked
out before
hand.

I believe that Hutter offered the prize in good faith. And I believe
the first few
successes offered their solutions with good faith and within the spirit
of the
competition. It would be wise to immediately suspend the competition
to prevent
any further winners until this is worked out.

For what it is worth, I think Shkarin's entry should be honored as a
prize winner
because he cleared the 1% mark above what was posted at the time of his
submission.
To do otherwise would be to invite the idea of 'delayed submissions' to
negate prize winning.
Not that I'm suggesting this is the case.

Also, removing the 1% improvement will eliminate this problem in the
future. And I doubt
that winners with little money gain will claim their prize.

At any rate, silence is a very bad idea now. Please ... suspend the
competition until the
rules are worked out. Anything else will become a very big problem.

Finally, I'm not surprised at all that Dmitry was the first 'winner'.
Even on your test, Matt, ppmonster is the clear 'winner' in my eyes.
(^:

- Michael Maniscalco



Matt Mahoney wrote:
Michael Goldshteyn wrote:
Why were the hutter prize entries removed from the Hutter prize page? There
is no longer a mention of the candidate entries at either:

http://prize.hutter1.net/

or

http://cs.fit.edu/~mmahoney/compression/text.html


Thanks,

Mike

There is some controversy over what the rules are and who won a prize,
if anyone. Marcus Hutter will make the final decision on both matters,
since it's his money. So I removed this information from my website.

The controversy stems from how much time and memory should be allowed.
My benchmark was vague on this and simply said that a program should
use "reasonable" resources, otherwise nobody will test the program and
it won't be listed. This was adequate for an open benchmark where
anyone can submit results, but not for one where prize money is
involved and we want to test the claims ourselves.

The current rules require that a decompressor use no more than 1 GB of
memory, 10 GB of temporary files and run in under 8 hours on a "1 GHz
machine". However, these rules were not stated at the time of the
submission. After these programs were submitted, I had to upgrade my
PC to 2 GB and install Linux (dual boot) to verify the two claims.
Both were valid at the time of submission, but neither would be now.

A second controversy is over who should win the prize if we do allow 2
GB. The two submissions were a few hours apart. raq8g was first, with
a claimed a 0.9% improvement using 1 GB (-7 option), which was below
the 1% threshold for a prize. (The author claimed the right amount but
mistakenly believed it was over 1%). I tested it with 2 GB (-8 option)
and found a 2.0% improvement (taking 2 x 9 hours on a 2.2 GHz Athlon 64
under 64 bit Ubuntu Linux). The controversy here is that the author
never claimed the higher amount because he did not test with 2 GB. The
new rules make it clear that we will only test as instructed by the
author and not grant more than the claimed amount. No such rule
existed at the time of submission. My policy was to experiment with
options, making a "reasonable" effort to improve compression. It is
reasonable to use as much memory as possible, since this almost always
helps.

durilca 0.5(Hutter) was submitted a few hours after raq8g. It improved
by 1.5% over the original amount using 2 GB, but by less than 1% using
1 GB. There was no claim for 1 GB, but since tests only took about 10
minutes, so I was able to experiment with options to get what I think
is the best result. So even if we allowed 2 GB, only one of the two
would win a prize.

Results for both can be found at
http://cs.fit.edu/~mmahoney/compression/text.html but you have to look
for them because they are not in the main table. That is for enwik9.
durilca 0.5(Hutter) crashes on enwik9 even with 2 GB. raq8g would take
too long to test, so I probably will not. However, you can find enwik8
results at 1 GB and 2 GB in the detailed descriptions for each program.

Unfortunately, this doesn't give you the decompressor size using the
same measurement as the Hutter prize. I allow a zipped archive
containing either source code (any language) or .exe, plus dictionaries
and other files. Hutter allows only a single uncompressed .exe (self
extractors like UPX are OK). Such rule incompatiblities are necessary
when money is involved.

This is a messy situation, unfortunately.

It is not over yet. I am testing a third submission that should
improve by 2.6% with 1 GB.

-- Matt Mahoney

.

Relevant Pages

  • Re: Hutter prize entries?
    ... There is some controversy over what the rules are and who won a prize, ... The controversy stems from how much time and memory should be allowed. ... Both were valid at the time of submission, ... never claimed the higher amount because he did not test with 2 GB. ...
    (comp.compression)
  • Re: At Long Last - The Limerick Coin Contest Winners
    ... Thanks for all the entries and I hope everyone had as much fun as I had. ... I'm pleased and flattered that my submission made the final cut. ... "I hereby withdraw my remaining submission in deference to the quality of ... You may opt out of the prize if you so wish but as far as ...
    (rec.collecting.coins)
  • Re: Remember Bex in Casualty?
    ... their collective head out of their asses, you could win the Booker ... prize with that submission. ... Little coward. ...
    (uk.media.tv.misc)
  • Re: Anyone want a K1000 / K2000 to play with? :-)
    ... such a shame it's in the UK, ... hope you're getting a good prize for it. ... Just need to come and collect and pay or Ad submission and it's yours! ...
    (comp.sys.tandem)
  • Re: Winners say "no thanks"...
    ... You want the amount of the prize after taxes to be 12000: ... The tax owed is a percentage of the actual prize, plus a percentage of the "extra money for taxes". ...
    (rec.arts.disney.parks)