Compression test on permutations

Hi all,

after some discussion in another thread, I got curious about how current "universal compressors" would perform on a very specific input.
(I hope the admin of maximumcompression.com reads this :))
Also, it would be interesting to see how a "dedicated" (i.e. built specifically for this input) compressor would perform.

You can either generate this file by your own, or you can email me and I will provide you with it. (you just need to convert it to a binary form, from the ASCII, comma separated, format I generated it in)

File description:

The file is made of 65536 bytes.
Each 256 bytes represent a pseudo-randomly ordered permutation of a byte. (i.e. of all the numbers represented by 8-bits)
So, basically you have 256 permutations of size 256-bytes each, one after the other.

Nothing can be assumed beforehand about the file - even though it's just pseudo-random, rather than truly random - but the following obvious considerations:

- Each 256 consecutive-bytes string always contains all the numbers from 0 to 255.
- There is no repeated 256-bytes string within the whole file
- There is no repeated byte within a whole 256-bytes string (permutation)

....and anything else comes to your mind about such a file.

It would be great to have feedback about this test...if it's been done before, if it's interesting...and of course, the results on the most known compressors! ;)


P.S. if anyone is willing to convert my ASCII file in Win32 binary form, I'd be grateful... It's far too much hassle to search for and compile a Lua library that handles binary files. (the binary mode in Lua is fundamentally fake... :))


Best,
E.
.

Relevant Pages

  • Re: random .. ?
    ... >> is no program shorter than your string that outputs that string, ... so all compressors should fail. ... I bet there are strings that could pass statistical tests but still ...
    (sci.crypt)
  • Re: New large text benchmark
    ... Matt Mahoney wrote: ... I am now up to 40 ranked compressors including contributions by others. ... use all available memory. ... it compress a string that string is gone and 2 strings 1 bit longer are put ...
    (comp.compression)
  • Re: New large text benchmark
    ... Matt Mahoney wrote: ... I am now up to 40 ranked compressors including contributions by others. ... use all available memory. ... it compress a string that string is gone and 2 strings 1 bit longer are put ...
    (comp.compression)
  • Re: random .. ?
    ... >>is no program shorter than your string that outputs that string, ... questionable for infinite strings. ... part of the program shorter than the infinite program which just prints the ... uncompressible by any compressors out there now, ...
    (sci.crypt)