Information Theory and a Conserved Substance During Signal Transmissions?

One may obtain equilibrium particle number distributions in statistical mechanics by applying time reversal balance to reactions which conserve energy. For example, one may obtain the Maxwell-Boltzmann distribution by using f(e1)f(e2)=f(e3)f(e4), taking ln of both sides and equating to: e1+e2=e3+e4. For more complicated situations, one may use “reaction probabilities” instead of particle number probabilities i.e. g(f(e1)) g(f(e2)) = g(f(e3)) g(f(e4)). Taking ln of both sides and equating to e1+e2=e3+e4 yields more complicated distributions such as the Fermi-Dirac or Bose-Einstein distribution for which g=f/(1-f) and g=f/(1+f). For the Maxwell-Boltzmann case, one may define Shannon’s entropy density  -f ln(f) which when varied with respect to f subject to the constraint 1/T ei f(ei) yields the MB distribution. For the more complicated case, one may define a Shannon’s entropy density for g(f(ei)) i.e. - g ln(g).

     In information theory, matters seem to be different i.e. one does not seem to have energy conserving interactions. For instance, one may start with a long message written in English for which there is a probability Pi for each letter to occur. The message is then transmitted and it is possible that an “a” in the original message is converted to a different letter in the received message. It seems, however, that Pi must remain the same in the received message. If this were not the case, the transmission would immediately be suspect. Given this constraint, it seems one may consider each initial letter undergoing interactions until it arrives at the reception point. If Pi is unchanged, it seems there is a conserved quantity in the system and we try to show that it is ln(Pi) i.e. information. 

    The situation of Pi remaining unchanged during “interactions” also applies to statistical mechanical equilibrium because one begins with f(ei) in a Maxwell-Boltzmann gas, and even though each particle may change its energy through numerous interactions, after time t, f(ei) must be the same for there to be equilibrium. We argue that this is the link between information theory and statistical mechanics. As far as we know, this view is not included in descriptions of information theory which compare it to statistical mechanics.