Maximum Entropy, Bias Removal, Conserved Quantity and Second Law of Thermodynamics

Given an unbiased die, the probability is ⅙ for a result. This probability value may be obtained directly by imposing the condition of a lack of bias and noting there are six possible results. Alternatively, one may maximize Shannon’s entropy subject to the constraint Sum over i p(i)=1 where i runs from 1 to 6. Maximization of Shannon’s entropy seems to remove as much bias as possible with the constraint indicating what bias remains. For the die case the constraint has no bias, but for an average energy constraint Sum over i ei p(i) there is, namely ei.

   In previous notes we have argued that the constraint used in the maximization of entropy is “special”. Here we argue that it represents a conserved quantity if interactions occur. Thus there may be many constraints in a system, but choosing a conserved quantity in an interaction is rarer, but seems to be linked with equilibration. Finally, we suggest that maximization of entropy subject to a conserved quantity constraint removes as much bias as possible and it is this maximized entropy-probability  which is subject to the second law of thermodynamics.

   It is possible, however, to have probabilities and to create entropies which are not based on maximization of entropy. For example, Shannon’s entropy may be applied to the spatial density of various n-slit interactions for a particle with constant momentum p. In this case, even though there is no maximization of entropy, the emerging particle has the same p magnitude (hence same uncertainty within a wavelength), but there is the added uncertainty of its direction. As a result, it seems that entropy increases in accordance with the second law.

  A free particle with momentum p, however, has a two dimensional probability distribution in space exp(ipx) characterized by a wavelength hbar/p. This may be taken as related to the entropy of the free particle, but a free particle may undergo spontaneous interactions which either increase or decrease p, hence increase or decrease hbar/p and entropy. In other words, the second law does not apply to such an entropy, but this entropy is associated with an unusual constraint. This conclusion seems to hold when one considers two body scattering. Thus we suggest that there are various kinds of entropy. Some are linked with maximization of entropy subject to a conserved quantity constraint (and are associated with equilibration) and others are not. The latter do not necessarily obey the second law of thermodynamics, we argue.