Why modern physics: the cracks in classical physics around 1900

At the end of the nineteenth century physics looked nearly finished. Newton's mechanics predicted the motions of the planets so well that Neptune had been found in 1846 by calculation before anyone saw it through a telescope. Maxwell's equations of 1865 united electricity, magnetism and light, and Hertz had produced and detected radio waves in 1887 exactly as the equations said he would. Thermodynamics and the kinetic theory of gases explained heat engines, the pressure of a gas and the viscosity of air. A student entering physics in 1890 could be forgiven for thinking that the remaining work was a matter of measuring constants to more decimal places.

Within thirty years that picture had been replaced by two new theories, relativity and quantum mechanics, which together form what we call modern physics. Neither theory threw classical physics away. Newton's laws remain exact enough to land a probe on a comet, and Maxwell's equations survived relativity untouched. What changed was our understanding of where classical physics stops working: at speeds close to the speed of light, and at the scale of atoms and below. This course follows that change, from the experiments that would not fit the old theories to the technologies built on the new ones, such as GPS receivers, lasers, PET scanners, MRI machines and the transistors inside every phone.

This first lesson surveys the cracks. Each one will become a lesson of its own, so the aim here is to see why each result was so troubling, what classical physics predicted, and how far the prediction was from the measurement.

Two clouds on the horizon

In April 1900 William Thomson, Lord Kelvin, gave a lecture at the Royal Institution in London titled Nineteenth-Century Clouds over the Dynamical Theory of Heat and Light. The theory itself was beautiful and clear, he said, but two clouds obscured it. The first was the failure of experiments to detect the motion of the Earth through the luminiferous ether, the medium that was supposed to carry light waves. The second was the failure of the equipartition theorem to give the right heat capacities of gases. Kelvin expected both clouds to blow away with better experiments. Instead the first became special relativity and the second became quantum theory.

Kelvin's list was incomplete. Several more puzzles were known or appeared within a few years: the spectrum of light from hot objects, the photoelectric effect, the sharp lines in the spectra of gases, the stability of atoms, and radioactivity. The table below lists the main ones, what classical physics expected, and what was observed.

The ether and the speed of light

Maxwell's equations predict electromagnetic waves travelling at \(c = 1/\sqrt{\mu_0 \varepsilon_0}\), about \(3.00 \times 10^8\ \mathrm{m/s}\), the measured speed of light. But speed relative to what? Sound travels at 343 m/s relative to the air, and a listener running toward the source hears it arrive faster. Physicists naturally assumed that light, too, was a wave in a medium, the ether, and that Maxwell's equations held exactly only in the rest frame of the ether. The Earth moves around the Sun at about 30 km/s, so for at least part of the year it should be moving through the ether, and light should travel at slightly different speeds in different directions.

In 1887 Albert Michelson and Edward Morley in Cleveland built an interferometer floating on a pool of mercury so that it could be rotated smoothly. Light was split into two beams travelling at right angles, reflected back and recombined. If the Earth moved through the ether, rotating the apparatus would shift the interference fringes by about 0.4 of a fringe. They saw a shift of less than 0.02 fringe, consistent with zero. Repeated in different seasons and refined many times since, the result has never changed: modern versions with optical cavities compare light speeds in different directions to about one part in \(10^{17}\). Lessons 2 and 3 show how Einstein resolved the puzzle by taking the result at face value.

Hot objects and the ultraviolet catastrophe

Everything glows. A stove element turns dull red near 800 K and a tungsten filament glows white near 2,800 K. By the 1890s precise measurements, especially at the Physikalisch-Technische Reichsanstalt in Berlin, had mapped the spectrum of a blackbody, an idealized object that absorbs all radiation falling on it. The spectrum rises to a peak and falls off, and the peak moves to shorter wavelengths as the temperature rises.

Classical physics gave a definite prediction for this spectrum. Count the standing-wave modes of the electromagnetic field in a box, give each mode an average energy \(kT\) according to equipartition, and you obtain the Rayleigh–Jeans law. It works at long wavelengths. At short wavelengths it predicts that the energy density grows without limit, so that a warm oven would contain infinite energy, most of it in the ultraviolet and beyond. Paul Ehrenfest later called this the ultraviolet catastrophe. Lesson 5 derives both the classical law and Max Planck's solution of 1900, in which energy is exchanged only in packets \(E = hf\).

Light that knocks out electrons

Hertz noticed in 1887 that ultraviolet light made sparks jump more easily across a gap. Philipp Lenard showed in 1902 that light ejects electrons from metal surfaces and found two disturbing facts. The kinetic energy of the ejected electrons did not depend on the intensity of the light, only on its frequency. And below a threshold frequency no electrons came out at all, however bright the light. A classical wave delivers energy in proportion to its intensity, so a bright enough red light should eventually shake electrons loose. It does not. Einstein's explanation in 1905, that light itself comes in quanta of energy \(hf\), is the subject of lesson 6.

Lines in the spectra of gases

Pass the light from a hydrogen discharge tube through a prism and you do not see a rainbow. You see a few sharp coloured lines. In 1885 Johann Balmer, a Swiss schoolteacher, found that the wavelengths of the four visible hydrogen lines fit a formula with integers in it. Johannes Rydberg generalized it in 1888 to

with the Rydberg constant \(R_\infty \approx 1.097 \times 10^7\ \mathrm{m^{-1}}\) and integers \(n_2 > n_1\). Balmer's series has \(n_1 = 2\). Integers in a formula for light were a deep mystery. Classical mechanics allows an orbiting charge to have any energy, and its radiation should form a continuous spectrum. The program below reproduces Balmer's four lines from the formula and compares them with the measured values in air.

R = 1.0967758e7          # Rydberg constant for hydrogen, 1/m (includes the reduced-mass correction)
measured = {3: 656.28, 4: 486.13, 5: 434.05, 6: 410.17}   # wavelengths in air, nm
n_air = 1.000277         # refractive index of air for visible light
names = {3: 'H-alpha (red)', 4: 'H-beta (blue-green)', 5: 'H-gamma (violet)', 6: 'H-delta (violet)'}
print(f"{'line':22s}{'vacuum nm':>11s}{'air nm':>9s}{'measured':>10s}")
for n2 in range(3, 7):
    inv = R * (1 / 2**2 - 1 / n2**2)
    lam_vac = 1e9 / inv
    lam_air = lam_vac / n_air
    print(f"{names[n2]:22s}{lam_vac:11.2f}{lam_air:9.2f}{measured[n2]:10.2f}")
series_limit = 1e9 / (R / 4)
print(f"series limit (n2 -> infinity): {series_limit:.1f} nm in vacuum")

line                    vacuum nm   air nm  measured
H-alpha (red)              656.47   656.29    656.28
H-beta (blue-green)        486.27   486.14    486.13
H-gamma (violet)           434.17   434.05    434.05
H-delta (violet)           410.29   410.18    410.17
series limit (n2 -> infinity): 364.7 nm in vacuum

The agreement is to better than a hundredth of a nanometre, which is far better than any classical model of the atom could manage. Balmer had found the formula by pure numerical pattern-finding, with no idea where the integers came from. Niels Bohr's model of 1913, lesson 9, explained them as labels of allowed energy levels.

Atoms that should not exist

Ernest Rutherford's scattering experiments, carried out by Hans Geiger and Ernest Marsden in Manchester and published by Rutherford in 1911, showed that nearly all the mass of an atom sits in a tiny positive nucleus about \(10^{-14}\) m across, with electrons somewhere around it in a region \(10^{-10}\) m across. The obvious picture is a miniature solar system. But an electron in a circular orbit is accelerating, and Maxwell's equations say that an accelerating charge radiates. The Larmor formula gives the radiated power as

As the electron loses energy it spirals inward, and the closer it gets, the faster it radiates. Integrating the energy loss from an orbit the size of a hydrogen atom down to the nucleus gives the time for the collapse. The program below does the integral numerically, in small steps of radius, and compares the result with the closed-form answer.

import math
e, me, eps0, c = 1.602176634e-19, 9.1093837e-31, 8.8541878128e-12, 2.99792458e8
k = 1 / (4 * math.pi * eps0)
r0 = 5.29177e-11          # starting radius, m (the size of a hydrogen atom)
# circular orbit: E = -k e^2 / (2 r), a = k e^2 / (me r^2)
def power(r):
    a = k * e**2 / (me * r**2)
    return e**2 * a**2 / (6 * math.pi * eps0 * c**3)
def dEdr(r):
    return k * e**2 / (2 * r**2)
# dt = dE / P = (dE/dr) dr / P ; integrate from r0 down to a tiny radius
N = 200000
t = 0.0
rmin = r0 * 1e-4
for i in range(N):
    ra = r0 - (r0 - rmin) * i / N
    rb = r0 - (r0 - rmin) * (i + 1) / N
    rm = 0.5 * (ra + rb)
    t += dEdr(rm) * (ra - rb) / power(rm)
exact = r0**3 * 4 * math.pi**2 * eps0**2 * me**2 * c**3 / e**4
print(f"numerical collapse time: {t:.3e} s")
print(f"closed form r0^3 * 4 pi^2 eps0^2 me^2 c^3 / e^4: {exact:.3e} s")
v0 = math.sqrt(k * e**2 / (me * r0))
print(f"starting orbital speed: {v0:.3e} m/s = {v0 / c:.4f} c")
print(f"starting orbital period: {2 * math.pi * r0 / v0:.3e} s")

numerical collapse time: 1.556e-11 s
closed form r0^3 * 4 pi^2 eps0^2 me^2 c^3 / e^4: 1.556e-11 s
starting orbital speed: 2.188e+06 m/s = 0.0073 c
starting orbital period: 1.520e-16 s

A classical hydrogen atom would collapse in about 16 picoseconds, after making roughly a hundred thousand orbits. Yet hydrogen atoms in the gas clouds of the Milky Way have lasted for more than ten billion years. This is not a small discrepancy to be fixed by adjusting a parameter; it is a factor of more than \(10^{28}\). Classical physics simply has no stable atoms. Quantum mechanics replaces the spiralling orbit by stationary states that do not radiate, lessons 8 and 9.

Heat capacities and radioactivity

The equipartition theorem says that in thermal equilibrium every quadratic term in the energy holds an average of \(\tfrac12 kT\). A diatomic molecule such as nitrogen has three translational and two rotational degrees of freedom, plus vibration, which contributes two more quadratic terms. The predicted molar heat capacity at constant volume is therefore \(\tfrac72 R\), yet nitrogen at room temperature gives \(\tfrac52 R\), as if the vibration did not exist, and hydrogen at 50 K gives \(\tfrac32 R\), as if it could not even rotate. Degrees of freedom seemed to freeze out at low temperature. Quantization explains this directly: a vibration whose energy steps are much larger than \(kT\) is almost never excited.

Radioactivity, discovered by Henri Becquerel in 1896 and studied by Marie and Pierre Curie, who isolated polonium and radium in 1898, was a different kind of crack. A gram of radium releases heat steadily, enough to melt its own mass of ice in about an hour, and goes on doing so for centuries without any visible fuel. No chemical reaction could do that. The source turned out to be the conversion of a little mass into energy inside the nucleus, \(E = mc^2\), which connects lesson 4 on relativistic energy to lesson 11 on the nucleus.

How the new theories fit with the old

Relativity and quantum mechanics share a pattern. Each has a parameter that measures how far a situation is from the classical world. For relativity it is the speed ratio \(\beta = v/c\); when \(\beta\) is small, the relativistic formulas reduce to Newton's. For quantum mechanics it is the size of Planck's constant compared with the action of the system, roughly the product of a typical momentum and a typical length; when that product is huge compared with \(h\), quantum effects average out and classical mechanics returns. This rule, that new theories must reproduce old ones in the domain where the old ones were tested, was stated clearly by Bohr as the correspondence principle. It is the reason you can trust Newton for bridges and orbits while knowing that he is not the final word.

It is also why the cracks appeared where they did: in measurements of light, which always travels at \(c\), and in measurements of atoms, where the action is a few times \(h\). Each later lesson will make this precise. For now it is worth remembering that the experimenters who found the cracks were not looking for revolutions. Michelson wanted to measure the Earth's motion, the Berlin group wanted better standards for lighting, Lenard wanted to understand cathode rays. Careful measurement, taken seriously when it disagreed with expectation, did the rest.