Derivation of Shannon's Entropy for Constraints Sum over i g(ei)P(ei) with Probability P(ei) Part 2

   In Part 1, we argued that if one has a conserved quantity g(ei), as given in a constraint Sum over i g(ei) P(ei), then one may solve for the probability distribution P(ei) = P1(g(ei)) by considering a subset of P1(g(ei))s such that g(ei)+g(ej) = E1. Within this subset, one has a uniform distribution for P1(g(ei))P1(g(ej))=P1(g(ei)+g(ej)) with P1 unnormalized and this leads directly to P1(g(ei)) = C exp(-g(ei)/T), i.e. the Maxwell-Boltzmann form. A uniform distribution implies a minimal amount of bias in keeping with the conservation of g(ei) which is equivalent to the global constraint Sum over i g(ei) P(ei). We then suggested that the subset-uniform distribution-conservation result of Cexp(-g(ei)/T) implies that ln(P1(g(ei)) = -g(ei)/T.

   In Part 1, we argued that this allows one to create a math function F(P1(g(ei)) which when maximized subject to the constraint Sum over i g(ei) P1(g(ei)) yields Cexp(-g(ei)/T). The idea of maximizing is justified, we argue because one already deals with a minimal amount of bias in the subset-conservation approach and so this minimal bias must carry over to the full P1(g(ei)) = P(ei) set subject to the constraint Sum over i g(ei) P1(g(ei). This led to the derivation of Shannon’s entropy (equivalent to F) in Part 1 as a math function. Physically, it is known that if one has  P(ei) and N particles, then NP(ei) = n(ei). As a result, one may physically permute the N particles creating  B= N!/ Product over i n(ei)! distinct arrangements. Now, there may be various different { n(ei) } sets which respect the constraint Sum over i g(ei) P1(g(ei)) = number, but yields different values for B. One wishes to have the maximum value of B as this represents minimal bias. Thus, we suggest that Shanon’s entropy should be linked with a physical scenario.

    The P(ei) = C exp(-g(ei)/T) solution is an approximate one as P(ei)P(ej) = P(ei+ej) may include ei+ej > E total, as is well-known. Given the uniform distribution from the subset-uniform distribution-conservation approach for which p(ei)p(ej)=p(ei+ej) ((1)), this result yields values for n(ei), n(ej) and n(ei+ej). Such a result may be compared with approximations of ln(n(ei))! (as one knows that approximations are being used). One finds that the common Stirling approximation is implied by ((1)) so that the Shannon’s entropy expression is equivalent to B in the Stirling approximation. Thus, Shannon’s entropy is linked to a physical global feature and is not simply a mathematical expression.