Re: Shannon's paper, and H as a lower bound on average code length.

From: John <gilbertj@xxxxxxxxx>
Date: Wed, 27 Aug 2008 08:00:04 -0700 (PDT)

On Aug 25, 2:19 pm, Thomas Richter <t...@xxxxxxxxxxxxxxxxx> wrote:

John schrieb:

Under certauincontraints he may have done, are you within the
constraints???

He shows for a given code that *that* code has H as a lower bound. But
what (from my understanding) he hasn't done is show that there is no
other code which can do better than H on average.

Your understanding is wrong.

Would you care to elaborate? Why does the statement:

"The converse part of the theorem, that C/H cannot be exceeded, may
be
proved by nothing that the entropy of the channel input per second is
equal to that of the source, since the transmitter must be non-
singular, and also this entropy cannot exceed the channel capacity.
Hence H' <= C and the number of symbols per second = H'/H <= C/H. "

establish H as the lowest bound on all uniquely decodable codes?

So long,
Thomas

Follow-Ups:
- Re: Shannon's paper, and H as a lower bound on average code length.
  - From: Thomas Richter
- Re: Shannon's paper, and H as a lower bound on average code length.
  - From: jacko

References:
- Shannon's paper, and H as a lower bound on average code length.
  - From: John
- Re: Shannon's paper, and H as a lower bound on average code length.
  - From: John
- Re: Shannon's paper, and H as a lower bound on average code length.
  - From: jacko
- Re: Shannon's paper, and H as a lower bound on average code length.
  - From: John
- Re: Shannon's paper, and H as a lower bound on average code length.
  - From: Thomas Richter

Prev by Date: Re: De-fading audio
Next by Date: Re: Zero motion vector Inter prediction in H264
Previous by thread: Re: Shannon's paper, and H as a lower bound on average code length.
Next by thread: Re: Shannon's paper, and H as a lower bound on average code length.
Index(es):
- Date
- Thread

Relevant Pages

Re: Shannons paper, and H as a lower bound on average code length.
... Your understanding is wrong. ... and also this entropy cannot exceed the channel capacity. ... Kraft inequality tells you that for a code to be uniquely decodable, ... where l_i is the length of the code of the i-th symbol. ...
(comp.compression)
Re: Disorderly Conduct
... What you perceive as my lack of understanding is ... entropy and you elect to parrot that. ... the spiritual universe), ... The original physical concepts, which are well understood by people ...
(talk.origins)
Re: Disorderly Conduct
... What you perceive as my lack of understanding is actually ... entropy and you elect to parrot that. ... the spiritual universe), ... The original physical concepts, which are well understood by people ...
(talk.origins)
Re: Is it true that when you mix two volumes of identical gas, there is no entropy change?
... In my understanding, entropy is number of states the system can be in. ... I did not know about the significance of knowing the number of ... molecules. ...
(sci.chem)