Why quantum mechanics: states, probabilities and measurement

Modern Physics, PHYS-271, told the story of how quantum ideas entered physics: Planck's quanta, Einstein's photons, Bohr's atom, de Broglie's waves and the first solutions of Schrödinger's equation. That course used quantum mechanics as a set of powerful rules. This course builds the theory itself. Over twelve lessons we will set up the wave function and the Schrödinger equation carefully, solve the standard problems exactly (the infinite well, the harmonic oscillator, the free particle, the delta function and finite wells, barriers, and finally the hydrogen atom), and develop the general formalism of states as vectors, observables as operators and measurement as projection. The level is that of a first full course in quantum mechanics, the material of the first four chapters of the standard textbooks, and the mathematics is the linear algebra of MATH-322 and the differential equations of MATH-201, used constantly.

Before any equations, this first lesson asks what is genuinely strange about quantum mechanics and what the theory is trying to describe. The answer can be put in three words: states, probabilities and measurement. A quantum state contains everything that can be known about a system, but it predicts the results of most experiments only as probabilities. Measuring a system generally changes its state. And states can be added, so that a system can be in a superposition of possibilities that, classically, would exclude each other. These ideas are easiest to see in a system with only two possible outcomes, so that is where we start.

The Stern–Gerlach experiment

In 1922 Otto Stern and Walther Gerlach, working in Frankfurt, sent a beam of silver atoms through a strongly nonuniform magnetic field. Each silver atom has a magnetic moment, from one unpaired electron, and the field gradient pushes the atom up or down by an amount proportional to the component of the moment along the field. Classically, the moments should point in random directions, so the beam should spread into a continuous smear on the collecting plate. Instead it split cleanly into two spots. Whatever direction the magnet is turned, the component of the moment along that direction takes only two values. We now call this property spin, and for the electron the two values of the spin component along any axis are \(+\hbar/2\) and \(-\hbar/2\), called spin up and spin down.

The Stern–Gerlach result alone is surprising. Sequences of such measurements are stranger. Suppose we pass the beam through a magnet oriented along \(z\) and block the down beam, keeping only atoms that gave \(+\hbar/2\) along \(z\). Send these through a second \(z\) magnet: all of them come out up. The measurement is repeatable. Now send them instead through a magnet oriented along \(x\). They split evenly, half up along \(x\) and half down, one atom at a time, apparently at random. Finally, keep only the \(x\)-up atoms and send them through a third magnet along \(z\) again. Classically they were all \(z\)-up a moment ago. Yet they split evenly once more. The \(x\) measurement has erased the \(z\) information. Measuring one component disturbs the other, and there is no state in which both have definite values.

States as vectors

Quantum mechanics describes this with a state vector. For the spin of one electron, every possible state is a combination of two basis states, which we write in Dirac's notation as \(|{\uparrow}\rangle\) and \(|{\downarrow}\rangle\), spin up and spin down along \(z\). A general state is

where \(a\) and \(b\) are complex numbers called probability amplitudes. If we measure the spin along \(z\), the probability of getting up is \(|a|^2\) and of getting down is \(|b|^2\). This is the Born rule, proposed by Max Born in 1926, in its simplest form: probabilities are the squared magnitudes of amplitudes. The normalization condition says the probabilities add to one. After the measurement the state is no longer \(|\chi\rangle\); it is \(|{\uparrow}\rangle\) if we found up and \(|{\downarrow}\rangle\) if we found down. That is why a second \(z\) measurement repeats the first.

The states of definite spin along \(x\) are particular superpositions of the \(z\) states:

An atom in \(|{\uparrow}\rangle\) can be rewritten as \((|{+x}\rangle + |{-x}\rangle)/\sqrt2\), so an \(x\) measurement gives each result with probability \(1/2\). An atom in \(|{+x}\rangle\) has \(z\) amplitudes of \(1/\sqrt2\) each, so a following \(z\) measurement is again an even split. The three-magnet puzzle is explained by nothing more than writing the same vector in two different bases. The relative sign between the components matters: \(|{+x}\rangle\) and \(|{-x}\rangle\) have identical \(z\) probabilities, yet they are completely different states, as an \(x\) measurement shows. Amplitudes carry phase information that probabilities alone do not.

This two-dimensional complex vector space is the simplest example of the structure the whole course uses. For a particle moving along a line, the state is a wave function \(\psi(x)\), which we can think of as a vector with one component for every point \(x\); the space of such functions is infinite-dimensional, but the rules are the same. Lesson 8 makes this precise, and the functional analysis course MATH-445 treats the mathematics of these spaces, Hilbert spaces, in depth.

Interference: why amplitudes, not probabilities

Why not simply describe a quantum system by a list of probabilities, as a gambler would describe a pair of dice? Because quantum probabilities do not add the way classical ones do. In the two-slit experiment of PHYS-271, an electron reaching a point on the screen could have come through either slit. If probabilities added, the pattern with both slits open would be the sum of the two single-slit patterns. Instead we add amplitudes, \(\psi = \psi_1 + \psi_2\), and then square:

The last term, the interference term, can be positive or negative. Where it is negative, opening a second slit reduces the number of electrons arriving, something no classical mixture of possibilities can do. Interference is the experimental signature of superposition, and it is why the theory needs complex amplitudes. When we ask which slit an electron went through, by any method that could in principle answer the question, the interference term vanishes and the probabilities add classically. Information about the path and interference are mutually exclusive.

What is uncertain, and what is not

It is tempting to read quantum probabilities as ignorance: perhaps each silver atom secretly has definite values of both spin components and we just do not know them. Einstein, Podolsky and Rosen argued in 1935 that quantum mechanics must be incomplete for exactly this reason. In 1964 John Bell showed that any theory of such hidden values that respects locality, the requirement that a measurement here cannot instantly affect a result far away, makes predictions for correlated pairs of particles that differ measurably from quantum mechanics. Experiments beginning with John Clauser in 1972 and Alain Aspect in 1982, and by 2015 free of the main loopholes, agree with quantum mechanics and violate Bell's inequality. Clauser, Aspect and Anton Zeilinger shared the 2022 Nobel Prize in Physics for this work. The probabilities are not a mask over hidden definite values of the local kind; they are part of nature as far as anyone has been able to test.

What is not uncertain is the state itself. Given the state, quantum mechanics predicts probabilities exactly, and between measurements the state evolves in a perfectly deterministic way, by the Schrödinger equation of lesson 3. The randomness enters only when a measurement selects one outcome. How, or whether, that selection really happens is the measurement problem, still debated in competing interpretations. For calculating what experiments will show, every interpretation uses the same rules, and those are what this course teaches.

The postulates in outline

The structure of the theory can be summarized in a few statements, which later lessons develop one at a time:

Everything in the course follows from these rules plus the specific operators for specific systems. The two-state spin system already contains all of them. The operator for the \(z\) component of spin is a 2 by 2 matrix with eigenvalues \(\pm\hbar/2\) and eigenvectors \(|{\uparrow}\rangle\) and \(|{\downarrow}\rangle\); the \(x\) component is another matrix with the same eigenvalues but eigenvectors \(|{\pm x}\rangle\). Because the two matrices do not commute, they have no common eigenvectors, which is the precise statement that \(S_x\) and \(S_z\) cannot both be definite. Lesson 9 turns this into the general uncertainty principle.

Simulating measurements

The program below plays the three-magnet experiment with a seeded random number generator. Each atom carries a two-component state vector; each magnet computes the Born probabilities in its own basis, picks an outcome at random, and replaces the state by the corresponding basis vector. Blocking a beam means keeping only one outcome. The counts approach the predicted fractions as the number of atoms grows, with the statistical scatter of order \(\sqrt N\) that any random sampling shows.

import math, random
rnd = random.Random(372)
s = 1 / math.sqrt(2)
BASES = {'z': [(1, 0), (0, 1)], 'x': [(s, s), (s, -s)]}   # up and down vectors

def measure(state, axis):
    """Return (outcome, new state): outcome 0 is up, 1 is down along the axis."""
    up, down = BASES[axis]
    amp_up = up[0].conjugate() * state[0] + up[1].conjugate() * state[1]
    p_up = abs(amp_up) ** 2
    return (0, up) if rnd.random() < p_up else (1, down)

N = 20000
kept_z, kept_x, final = 0, 0, [0, 0]
for _ in range(N):
    state = (complex(s), complex(s))          # atoms enter in an arbitrary state
    k, state = measure(state, 'z')
    if k: continue                            # block z-down
    kept_z += 1
    k, state = measure(state, 'x')
    if k: continue                            # block x-down
    kept_x += 1
    k, state = measure(state, 'z')
    final[k] += 1
print(f"atoms in: {N}, passed first z magnet (up): {kept_z}")
print(f"passed x magnet (up): {kept_x}  fraction {kept_x / kept_z:.3f}")
print(f"third magnet along z: up {final[0]}, down {final[1]}, "
      f"fraction up {final[0] / kept_x:.3f}")
print(f"expected 1 sigma scatter in that fraction: {math.sqrt(0.25 / kept_x):.3f}")

atoms in: 20000, passed first z magnet (up): 9963
passed x magnet (up): 4956  fraction 0.497
third magnet along z: up 2490, down 2466, fraction up 0.502
expected 1 sigma scatter in that fraction: 0.007

The third magnet splits the beam evenly even though every atom entering the \(x\) magnet had been found spin up along \(z\). Change the middle magnet to \(z\) in the program and every atom comes out up at the end: a repeated measurement of the same quantity gives the same answer. The simulation is a faithful model of the experiment, but notice what it cannot do: it cannot assign each atom definite values of both components in advance. Bell's theorem shows that for entangled pairs no local model of that kind can reproduce all the quantum predictions.

Where quantum mechanics is needed

Classical mechanics works when the action of a motion, roughly momentum times distance or energy times time, is huge compared with Planck's constant \(\hbar = 1.055 \times 10^{-34}\) J s. For a thrown ball the ratio is about \(10^{34}\) and quantum effects are invisible. For an electron in an atom it is of order one, and quantum mechanics is unavoidable. The theory is needed for atoms and molecules and therefore all of chemistry; for solids, where it explains why copper conducts and glass does not, and makes transistors, LEDs and lasers possible; for nuclei and particles; and for the newest technologies built on superposition and entanglement directly, such as atomic clocks, magnetic resonance imaging and quantum computers. In Canada, the Perimeter Institute and the Institute for Quantum Computing in Waterloo, and companies such as D-Wave in Burnaby, work at that frontier; in the United States, national quantum initiatives fund similar efforts at universities and laboratories including Fermilab and NIST.

The rest of the course follows a deliberate path. Lessons 2 to 7 work with wave functions on a line, building intuition with exactly solvable problems. Lessons 8 and 9 step back to the general formalism, where those examples become instances of linear algebra. Lessons 10 to 12 move to three dimensions, where angular momentum appears and the hydrogen atom can be solved exactly, the triumph of the theory in 1926 and its first great test.