Re: I know this won't work, I just want to know -why-.

dalek_no2@xxxxxxxxxxx wrote: 
Another crackpot thought pattern from me.  I occasionally poke my head
into this group and rattle off some inane, but unusual thought
regarding compression methods.  I understand the basic concepts of
compression, and I get the whole "you can't get something for nothing"
line of reasoning, but I still like thinking towards the more exotic
forms of data compression.

Anyway, recently, I found myself reasoning thusly:

It is possible, nay trivial, to create redundancy data for any file
which will detect and correct any error up to the size of the
redundancy data.  Simple, yes?

Why, therefore, would it be impossible to take a file, create just over
50% redundancy for it, delete the original file, create a -completely
random- file of exactly the same size, and apply the recovery data to
the random file?  Statistically, only about half of the bits should be
wrong.  Also, you're not compressing N-bytes into fewer than N-bytes,
since the random file is N-bytes, and the recovery data is 1/2 * N
bytes.

I suspect that my misunderstanding here lies in my understanding of
error detection and recovery, but I just wanted to make sure.


I am wondering about this error correction thingy all the time. I just can't believe it is really able to correct anything. I hadn't yet got the time to learn details about it or to run the huge amount of necessary tests to see it clearly, so please excuse, that what follows is what I believe, not what I know - inspite of this I hope i can be helpful here:

In my opinion the error correction implemented in many archiving programs works only in case the errors detected are errors with a very well known characteristics determined from experience with characteristics of failures found on storage media. In other words, if the kind of error is not as expected, the corrected version of the file will be wrong and there is no chance to see it when the original file version is no more available for comparison.

Sending files larger than 2 GByte over a 10 MBit network I have already seen many times damaged files on the receiving side without any notice about problems during transmission - theoretically as good as not possible, but seen by me multiple times (I was not able to reproduce the behaviour, since if I 'succeeded' in getting file damaged over the network again, the wrong data were not in the same area of it as before). With newer network (100 MBit and 1000 MBit) versions the problem with transmitting large files seem to go away, but maybe to provoke errors it need only larger files (e.g. 200 GByte)?

Anyone here who has seen files transmitted over the network which got damaged due to the way network transmits them and not due to detected problems during transmission or failures on storage media?

Claudio
.

Relevant Pages

  • Re: I know this wont work, I just want to know -why-.
    ... I am wondering about this error correction thingy all the time. ... Sending files larger than 2 GByte over a 10 MBit network I have already ... about problems during transmission - theoretically as good as not ...
    (comp.compression)
  • Digital Networks responds to Hyper-Ethernet
    ... Hyper-Ethernet enables the transmission of people. ... local area network since 1980, ... facility-transportation across inter-node links, DEC's latest DECNet ...
    (comp.os.vms)
  • Re: Lots of noise about user agent strings
    ... it should be measured as bits transmitted by the modem after the ... addition of network stuff and compression. ... modems are optimised to compress text for transmission and so ... read explanations how to do this with the user agent string. ...
    (comp.lang.javascript)
  • Re: Reporting a NetSurf error..?
    ... Bill wrote: ... me ..but, as mentioned, what happens "upon transmission" ... I'd agree with the point that where compression does ...
    (comp.sys.acorn.apps)
  • Re: Best way to back up a slave harddisk?
    ... It is possible to select varying degrees of compression when creating the ... Another alternative, if you are connected to a network, is to use network ... the laptop Microsoft's free power toy Sync toy checks for changes and keeps ... the main comp in sync with the external and the laptop uses the same to keep ...
    (uk.comp.homebuilt)