Entropy, temperature and the microcanonical ensemble

Lesson 2 showed that when two systems share energy, the division that maximizes the product of their multiplicities is overwhelmingly the most probable. This lesson turns that observation into thermodynamics. We define entropy as the logarithm of the multiplicity, find that the condition for maximum entropy is equality of a derivative that we can identify with inverse temperature, and then do the same for volume and particle number to obtain pressure and chemical potential. The isolated system with fixed energy, volume and particle number, in which every accessible microstate has the same probability, is called the microcanonical ensemble. Its whole thermodynamics follows from one function, \(S(U, V, N)\). The highlight of the lesson is a complete derivation of the entropy of a monatomic ideal gas, the Sackur–Tetrode formula, which PHYS-261 quoted without proof.

Entropy as a logarithm

Boltzmann defined the entropy of a macrostate as

where \(k = 1.380649 \times 10^{-23}\) J/K is Boltzmann's constant, exact since the 2019 redefinition of the SI. The logarithm has two virtues. First, it makes entropy additive: for two independent systems \(\Omega = \Omega_A\Omega_B\), so \(S = S_A + S_B\), as thermodynamics requires. Second, it turns the astronomically large multiplicities into manageable numbers. A mole of argon at room temperature has \(S \approx 155\) J/K, which means \(\ln\Omega \approx 1.1 \times 10^{25}\) and \(\Omega \approx 10^{4.9 \times 10^{24}}\).

With this definition, the statement that the most probable macrostate dominates becomes the second law: an isolated system evolves toward the macrostate of maximum entropy and, once there, stays. The entropy of the combined system in lesson 2 is \(S_A(U_A) + S_B(U - U_A)\), and it is maximized when

Temperature

The quantity that becomes equal in equilibrium is \(\partial S/\partial U\). Thermodynamics tells us that the quantity that becomes equal when two systems exchange energy is temperature, and that heat flows from hot to cold. If \(\partial S_A/\partial U_A < \partial S_B/\partial U_B\), transferring a little energy from A to B lowers \(S_A\) by less than it raises \(S_B\), so the total entropy increases and the transfer happens spontaneously: energy flows from the system with the smaller \(\partial S/\partial U\). Heat flows from hot to cold, so the smaller derivative must belong to the hotter system. The definition that fits, and that agrees with the ideal-gas temperature scale, is

Temperature measures how willing a system is to give up energy. A system whose entropy rises steeply with energy, a cold one, takes energy readily; a system whose entropy is nearly flat, a hot one, gives it away readily.

For the Einstein solid in the high-energy limit, \(S = Nk\ln(eq/N)\) with \(U = q\varepsilon\), so \(1/T = Nk/U\), which is \(U = NkT\): equipartition, with \(\frac12 kT\) for each of the two quadratic terms of each oscillator. At any energy the large-\(N\) formula from lesson 2 gives

which is Einstein's 1907 result. Differentiating \(U\) with respect to \(T\) gives the heat capacity, which falls to zero at low temperature because the oscillators freeze into their ground states.

A numerical temperature

The first part of the program below computes \(S = k\ln\Omega\) exactly for an Einstein solid of 5,000 oscillators and finds the temperature from a finite-difference slope, then compares with Einstein's formula. Even for this small solid the two agree to a few parts in ten thousand.

import math

# Part 1: temperature of an Einstein solid from S(U), compared with the exact formula
N = 5000                    # oscillators
def S(q):                   # entropy in units of k
    return math.lgamma(q + N) - math.lgamma(q + 1) - math.lgamma(N)
print("Einstein solid, N = 5000: temperature from the slope of S(U)")
for q in (500, 2000, 5000, 20000):
    slope = (S(q + 1) - S(q - 1)) / 2          # dS/dq, i.e. eps/(kT)
    kT_num = 1 / slope
    kT_exact = 1 / math.log(1 + N / q)          # large-N result in units of eps
    print(f"  q = {q:>5}: kT/eps from slope {kT_num:.5f}, from formula {kT_exact:.5f}")

# Part 2: Sackur-Tetrode entropy of the noble gases at 298.15 K and 1 bar
k = 1.380649e-23; h = 6.62607015e-34; NA = 6.02214076e23; u = 1.66053907e-27
T, P = 298.15, 1.0e5
measured = {'He': (4.0026, 126.15), 'Ne': (20.180, 146.33), 'Ar': (39.948, 154.85),
            'Kr': (83.798, 164.09), 'Xe': (131.29, 169.69)}
print("\nSackur-Tetrode molar entropy at 298.15 K, 1 bar (J/(mol K))")
for gas, (A, Smeas) in measured.items():
    m = A * u
    lam = h / math.sqrt(2 * math.pi * m * k * T)      # thermal wavelength
    v = k * T / P                                      # volume per atom
    s = NA * k * (math.log(v / lam ** 3) + 2.5)
    print(f"  {gas}: thermal wavelength {lam * 1e12:6.2f} pm, V/(N lambda^3) = {v / lam ** 3:.3e}, "
          f"S = {s:6.2f}, tabulated {Smeas:6.2f}")

# Part 3: mixing two gases, distinguishable or not (Gibbs paradox)
R = NA * k
print("\nMixing 1 mol + 1 mol at the same T and P")
print(f"  different gases: Delta S = 2 R ln 2 = {2 * R * math.log(2):.3f} J/K")
print(f"  same gas, with the 1/N! in Omega: Delta S = 0")
# without 1/N!, entropy of N atoms in V would be N k ln V + ...: removing a partition doubles V for each
print(f"  same gas, counting atoms as labelled (no 1/N!): Delta S = {2 * R * math.log(2):.3f} J/K, a paradox")

Einstein solid, N = 5000: temperature from the slope of S(U)
  q =   500: kT/eps from slope 0.41722, from formula 0.41703
  q =  2000: kT/eps from slope 0.79844, from formula 0.79824
  q =  5000: kT/eps from slope 1.44301, from formula 1.44270
  q = 20000: kT/eps from slope 4.48232, from formula 4.48142

Sackur-Tetrode molar entropy at 298.15 K, 1 bar (J/(mol K))
  He: thermal wavelength  50.54 pm, V/(N lambda^3) = 3.189e+05, S = 126.15, tabulated 126.15
  Ne: thermal wavelength  22.51 pm, V/(N lambda^3) = 3.610e+06, S = 146.33, tabulated 146.33
  Ar: thermal wavelength  16.00 pm, V/(N lambda^3) = 1.006e+07, S = 154.85, tabulated 154.85
  Kr: thermal wavelength  11.04 pm, V/(N lambda^3) = 3.055e+07, S = 164.09, tabulated 164.09
  Xe: thermal wavelength   8.82 pm, V/(N lambda^3) = 5.991e+07, S = 169.68, tabulated 169.69

Mixing 1 mol + 1 mol at the same T and P
  different gases: Delta S = 2 R ln 2 = 11.526 J/K
  same gas, with the 1/N! in Omega: Delta S = 0
  same gas, counting atoms as labeled (no 1/N!): Delta S = 11.526 J/K, a paradox

Negative temperature

The two-state paramagnet has a bounded energy. Its entropy is zero when all dipoles align with the field (lowest energy) and zero again when all point against it (highest energy), with a maximum of \(Nk\ln 2\) halfway. On the high-energy side, \(\partial S/\partial U\) is negative, so \(T\) is negative. A negative-temperature system is not colder than absolute zero; it is hotter than any positive temperature, since it will give energy to any system at positive \(T\). Such states were first made with nuclear spins in lithium fluoride by Edward Purcell and Robert Pound in 1951, as PHYS-261 described. They require a system with an upper bound on its energy and weak coupling to everything else, conditions met by spins but not by ordinary matter, whose kinetic energy is unbounded.

Pressure and chemical potential

The same argument works for any quantity two systems can exchange. If a movable piston separates them, they exchange volume, and the entropy is maximized when \(\partial S/\partial V\) is equal on both sides. Mechanical equilibrium requires equal pressure, and the definition that matches the ideal gas law is

If the systems exchange particles, equilibrium requires equal \(\partial S/\partial N\), and this defines the chemical potential:

The minus sign makes particles flow from high \(\mu\) to low \(\mu\), just as energy flows from high \(T\) to low \(T\). Collecting the three derivatives gives the thermodynamic identity,

the master equation of thermodynamics, now derived from counting. Lesson 7 builds a whole ensemble around the chemical potential.

The entropy of an ideal gas

For \(N\) identical atoms of mass \(m\) in a box of volume \(V\) with total energy \(U\), we count microstates semiclassically. Each atom has a position anywhere in the box and a momentum \(\mathbf p\); the energy is \(U = \sum|\mathbf p_i|^2/2m\). Quantum mechanics says each single-particle state occupies a phase-space volume \(h^3\), so the number of microstates is the accessible phase-space volume divided by \(h^{3N}\), and divided by \(N!\) because the atoms are identical:

The position integral gives \(V^N\). The momentum integral covers the thin shell where \(\sum p_i^2 = 2mU\), a sphere of radius \(\sqrt{2mU}\) in \(3N\) dimensions. The surface area of a sphere of radius \(r\) in \(d\) dimensions is \(2\pi^{d/2}r^{d-1}/\Gamma(d/2)\), and multiplying by a small shell thickness gives a factor that, after taking a logarithm, is negligible compared with terms of order \(N\). Keeping only those terms,

Taking the logarithm with Stirling's approximation, \(\ln N! \approx N\ln N - N\) and \(\ln(3N/2)! \approx \frac{3N}{2}\ln\frac{3N}{2} - \frac{3N}{2}\), and collecting terms gives the Sackur–Tetrode equation, found independently by Otto Sackur and Hugo Tetrode in 1912:

Using \(U = \frac32 NkT\), it takes a cleaner form in terms of the thermal de Broglie wavelength \(\lambda = h/\sqrt{2\pi mkT}\):

Every thermodynamic property follows. Differentiating with respect to \(U\) gives \(1/T = \frac32 Nk/U\), so \(U = \frac32 NkT\); differentiating with respect to \(V\) gives \(P/T = Nk/V\), which is \(PV = NkT\). The ideal gas law has been derived from counting states. The second part of the program evaluates the formula for the five stable noble gases at 298.15 K and 1 bar. The tabulated standard entropies are themselves computed this way, and calorimetric measurements, which integrate \(C_P/T\) from near absolute zero through melting and boiling, agree with them within experimental error. The quantity \(V/N\lambda^3\) is about \(10^5\) to \(10^7\): there is vastly more room per atom than the volume of a thermal wavelength, which is the condition for classical statistics to work, as lesson 8 will show.

The Gibbs paradox and the factor N!

Without the \(1/N!\), the entropy would contain \(Nk\ln V\) instead of \(Nk\ln(V/N)\), and it would not be extensive: doubling a system (both \(N\) and \(V\)) would more than double its entropy. The symptom is the Gibbs paradox. Remove a partition between two samples of the same gas at the same temperature and pressure. Nothing observable happens, so the entropy should not change. With labeled atoms the calculation gives the mixing entropy \(2R\ln 2 = 11.5\) J/K for two moles, exactly as if the gases were different, which they are not. Including \(1/N!\) removes the paradox. Gibbs introduced the factor in 1902 by an argument about what counts as a distinct state; quantum mechanics justifies it, since identical particles are fundamentally indistinguishable. The last part of the program prints the comparison.

The third law

As \(T \to 0\) a system settles into its lowest-energy state. If that state is unique, \(\Omega = 1\) and \(S = 0\). This is the statistical form of the third law of thermodynamics, stated by Walther Nernst in 1906. Some systems retain a residual entropy because their ground state is degenerate or because they freeze in disorder: ordinary ice has a residual entropy of about 3.4 J/(mol K), close to Linus Pauling's 1935 estimate \(R\ln(3/2)\) from counting hydrogen-bond arrangements. The Sackur–Tetrode formula fails at low temperature, giving a negative entropy, because the classical counting is invalid once \(\lambda^3\) becomes comparable to \(V/N\); the quantum gases of lessons 8 to 10 take over.