Re: Is evolution an example of decreasing entropy?


wade wrote:
> Robert J. Kolker wrote:
> > wade wrote:
> > >
> > > Informatic entropy is seriously misnamed and not associated with
> > > the 2nd law of thermodynamics.
> > >
> >
> > Statistical entropy and thermodynamic entropy are one and the same.
> >
> > See:
> >
> > http://www.nyu.edu/classes/tuckerman/stat.mech/lectures/lecture_6/node3.html
>
> They are decidedly not one in the same.

You are both wrong. Thermodynamic and statistical entropy are not "one
and the same", but they are deeply and intimately related.

Statistical entropy is a mathematical concept - to any probability
distribution, one may assign an "entropy", given by the Gibbs-Shannon
formula. When the probability distribution under consideration is the
distribution of a physical system over its possible microstates, the
corresponding statistical entropy is precisely the entropy as given by
statistical _mechanics_ - which in the limit of a large system becomes
equal to the thermodynamic entropy, as shown in the Tuckerman lecture
that Kolker cites.

To repeat:

The Entropy of Statistics is a mathematical concept.

The Entropy of Statistical Mechanics is a physical concept - an
application of statistical
entropy to a certain class of physical systems that are described in
statistical language.

The Entropy of Thermodynamics is a physical concept, which emerges from
the entropy of statistical mechanics when the probability distribution
collapses to a peak in accordance with the law of large numbers.

The Entropy of Statistics does not obey the Second Law of
Thermodynamics or any other physical law, since it is not a physical
concept.

The Entropy of Statistical Mechanics obeys the Second Law
_approximately_ (it is subject to fluctuations, whose magnitude is
proportional to the square root of the number of particles in the
system.)

The Entropy of Thermodynamics obeys the Second Law exactly, since the
fluctuations vanish in the limit of a macroscopic system.

In another post in this thread, Wade said:

"Informatic entropy is seriously misnamed and not associated with
the 2nd law of thermodynamics."

While the entropy as used in information theory - another application
of statistical entropy need not obey the 2nd Law - and in the class of
problems studied by Shannon, it does not - it does NOT follow that it
was "misnamed." Shannon and Von Neumann knew exactly what they were
doing when they chose their nomenclature. The connection between
thermodynamics and information theory goes back to Szilard in the
1920's. It was most extensively explored by E. T. Jaynes and his
successors, who reformulated the whole of statistical mechanics using
the language of information theory:

"The essential point... is that we accept the von Neumann-Shannon
expression for entropy, very literally, as a measure of the amount
of uncertainty represented by a probability distribution; thus entropy
becomes the primitive concept with which we work, more fundamental
even than energy. If in addition we reinterpret the prediction problem
of statistical mechanics in the subjective sense, we can derive the
usual relations in a very elementary way without any consideration of
ensembles or appeal to the usual arguments concerning ergodicity or
equal a priori probabilities."
[E. T. Jaynes, Information Theory and Statistical Mechanics, _Phys.
Rev._ _106_, 620 (1957),]

Wade again:
> Thermodynamic entropy is a real thing that has real units associated with it
> and these units include a temperature dependence as was mentioned in the part of my
> post you deceiptfully deleted.

Baloney. The fact that thermodynamic entropy, as conventionally
defined, has units is nothing more than a historical artifact, arising
from the arbitrariness of our temperature units. If we choose to
measure temperature in energy units (as is done in many textbooks on
Statistical Mechanics written by physicists, most notably the
celebrated treatise by Landau and Lifshitz), entropy becomes a
dimensionless quantity. In fact, the most rigorous treatments of
thermodynamics (such as presented in the books by Callen and by
Landsberg) regard entropy as a fundamental quantity, and *define*
temperature in terms of it.

Wade is right to draw a distinction between thermodynamic entropy (a
physical concept) and statistical entropy (a mathematical concept,
which becomes equal to the former when the mathematics is applied to a
particular class of physical systems) but he is wrong in attributing
any sort of significance to this business of units. It's a red herring.
Kolker is right to insist upon the equivlance (for macroscopic systems)
between the entropy of statistical mechanics and the entropy of
thermodynamics, but incorrectly conflates the entropy of statistical
mechanics with the general, mathematical concept of statistical
entropy.

--------
Robert

.

Loading