Spectral theory of compact self-adjoint operators

The spectral theorem for real symmetric matrices says that every such matrix has an orthonormal basis of eigenvectors, so that in suitable coordinates it is a diagonal matrix of real numbers. In infinite dimensions, bounded self-adjoint operators need not have any eigenvectors at all. Compact self-adjoint operators do, and for them the finite-dimensional theorem survives almost unchanged: they are diagonal in an orthonormal basis, with real eigenvalues tending to zero. This is the theorem Hilbert proved for symmetric integral equations between 1904 and 1906, and it explains why vibrating strings, drums and quantum particles in a box have discrete sets of frequencies and energies. This lesson defines the spectrum of a bounded operator, computes it for the basic examples, proves the spectral theorem for compact self-adjoint operators, and applies it to the eigenfunctions of the string.

The spectrum

Definition 11.1. Let \(T\) be a bounded operator on a complex Banach space \(X\). The resolvent set \(\rho(T)\) is the set of \(\lambda \in \mathbb{C}\) for which \(T - \lambda I\) is a bijection of \(X\), hence has a bounded inverse by the inverse mapping theorem. The spectrum is \(\sigma(T) = \mathbb{C} \setminus \rho(T)\). The point spectrum \(\sigma_p(T)\) is the set of eigenvalues, the \(\lambda\) for which \(T - \lambda I\) is not injective.

In finite dimensions an operator is injective exactly when it is bijective, so the spectrum consists of eigenvalues. In infinite dimensions, \(T - \lambda I\) can be injective without being onto, and then \(\lambda\) lies in the spectrum without being an eigenvalue. Theorem 11.2. The spectrum of \(T \in B(X)\) is a compact subset of the disc \(\{|\lambda| \le \|T\|\}\). Proof: if \(|\lambda| > \|T\|\), then \(T - \lambda I = -\lambda(I - T/\lambda)\) is invertible by the Neumann series. If \(\lambda_0 \in \rho(T)\), then by Corollary 3.6 \(T - \lambda I\) stays invertible for \(\lambda\) near \(\lambda_0\), so \(\rho(T)\) is open and \(\sigma(T)\) is closed. \(\blacksquare\)

Israel Gelfand proved in 1941 that the spectrum of every bounded operator on a nonzero complex Banach space is nonempty: if it were empty, the resolvent \(\lambda \mapsto (T - \lambda I)^{-1}\) would be an entire function tending to \(0\) at infinity, and applying functionals and Liouville's theorem shows it would vanish identically, which is absurd. He also proved the spectral radius formula \(r(T) = \max\{|\lambda| : \lambda \in \sigma(T)\} = \lim_n \|T^n\|^{1/n}\). Over the real numbers the spectrum can be empty, as for a rotation of the plane, which is why spectral theory is done over \(\mathbb{C}\).

Examples. (a) Multiplication by \(x\) on \(L^2[0, 1]\): for \(\lambda \notin [0, 1]\), the inverse of \(M - \lambda\) is multiplication by the bounded function \(\frac{1}{x - \lambda}\). For \(\lambda \in [0, 1]\), \((M - \lambda)u = (x - \lambda)u\) can be small with \(\|u\| = 1\), by concentrating \(u\) near \(\lambda\), so \(M - \lambda\) has no bounded inverse. Thus \(\sigma(M) = [0, 1]\), yet \(M\) has no eigenvalues, since \((x - \lambda)u = 0\) almost everywhere forces \(u = 0\). (b) The left shift \(S^*\) on \(\ell^2\): every \(\lambda\) with \(|\lambda| < 1\) is an eigenvalue, with eigenvector \((1, \lambda, \lambda^2, \ldots)\). Since \(\|S^*\| = 1\) and the spectrum is closed, \(\sigma(S^*)\) is the closed unit disc. The right shift \(S\) has the same spectrum but no eigenvalues at all. (c) The Volterra operator \(V\) on \(L^2[0, 1]\) satisfies \(\|V^n\| \le \frac{1}{n!}\), so its spectral radius is \(\lim (n!)^{-1/n} = 0\) and \(\sigma(V) = \{0\}\); and \(0\) is not an eigenvalue, since \(\int_0^x u = 0\) for all \(x\) forces \(u = 0\).

The spectrum of a compact operator

Theorem 11.3 (Riesz, 1918). Let \(K\) be a compact operator on an infinite-dimensional Hilbert space. Then \(0 \in \sigma(K)\); every nonzero \(\lambda \in \sigma(K)\) is an eigenvalue of finite multiplicity; and \(\sigma(K)\) is countable, with \(0\) its only possible accumulation point.

Proof. A compact operator on an infinite-dimensional space is not invertible, so \(0 \in \sigma(K)\). For \(\lambda \ne 0\), \(K - \lambda I = -\lambda(I - K/\lambda)\), and by the Fredholm alternative this is invertible unless it has nonzero kernel, which is then finite dimensional. For the last claim, suppose \(\lambda_1, \lambda_2, \ldots\) are distinct eigenvalues with \(|\lambda_n| \ge \delta > 0\) and eigenvectors \(u_n\). Eigenvectors for distinct eigenvalues are linearly independent, so the spaces \(M_n = \operatorname{span}\{u_1, \ldots, u_n\}\) increase strictly. Choose unit vectors \(y_n \in M_n\) orthogonal to \(M_{n-1}\). Since \((K - \lambda_n)M_n \subseteq M_{n-1}\), for \(m < n\) we get \(Ky_n - Ky_m = \lambda_n y_n + \big((K - \lambda_n) y_n - K y_m\big)\) with the bracket in \(M_{n-1}\), so \(\|Ky_n - Ky_m\| \ge |\lambda_n| \ge \delta\), contradicting compactness. So for each \(\delta\) only finitely many eigenvalues satisfy \(|\lambda| \ge \delta\). \(\blacksquare\)

Self-adjoint operators

Proposition 11.4. Let \(T\) be a bounded self-adjoint operator on a complex Hilbert space. (a) Every eigenvalue is real, and eigenvectors for distinct eigenvalues are orthogonal. (b) \(\sigma(T) \subseteq \mathbb{R}\). (c) \(\|T\| = \sup_{\|x\| = 1} |\langle Tx, x \rangle|\).

Proof. (a) If \(Tx = \lambda x\) with \(\|x\| = 1\), then \(\lambda = \langle Tx, x \rangle\), which is real by Proposition 6.4. If also \(Ty = \mu y\) with \(\mu \ne \lambda\), then \(\lambda \langle x, y \rangle = \langle Tx, y \rangle = \langle x, Ty \rangle = \mu \langle x, y \rangle\), so \(\langle x, y \rangle = 0\). (b) Let \(\lambda = \alpha + i\beta\) with \(\beta \ne 0\). For every \(x\), \(\|(T - \lambda)x\|^2 = \|(T - \alpha)x\|^2 + \beta^2 \|x\|^2 \ge \beta^2 \|x\|^2\), because the cross terms cancel by self-adjointness of \(T - \alpha\). So \(T - \lambda\) is injective, and the same lower bound shows its range is closed: if \((T - \lambda)x_n\) is Cauchy, so is \((x_n)\), and its limit \(x\) satisfies \((T - \lambda)x = \lim (T - \lambda)x_n\). The range is dense, since its orthogonal complement is \(\ker(T - \overline{\lambda}) = \{0\}\) by the same estimate. So \(T - \lambda\) is bijective. (c) Let \(m = \sup_{\|x\| = 1} |\langle Tx, x \rangle|\); clearly \(m \le \|T\|\). For unit vectors \(x, y\), expanding and using self-adjointness, \[4 \operatorname{Re} \langle Tx, y \rangle = \langle T(x + y), x + y \rangle - \langle T(x - y), x - y \rangle \le m(\|x + y\|^2 + \|x - y\|^2) = 4m,\] by the parallelogram law. With \(y = Tx / \|Tx\|\) this gives \(\|Tx\| \le m\). \(\blacksquare\)

Part (c) fails for operators that are not self-adjoint: the nilpotent matrix \(\begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}\) has norm \(1\), while its quadratic form has supremum \(\frac{1}{2}\). For self-adjoint operators it is the bridge between the norm and the eigenvalues.

Theorem 11.5. If \(T \ne 0\) is compact and self-adjoint, then \(\|T\|\) or \(-\|T\|\) is an eigenvalue of \(T\).

Proof. By Proposition 11.4(c) there are unit vectors \(x_n\) with \(\langle Tx_n, x_n \rangle \to \lambda\), where \(\lambda = \pm \|T\|\). Then \[\|Tx_n - \lambda x_n\|^2 = \|Tx_n\|^2 - 2\lambda \langle Tx_n, x_n \rangle + \lambda^2 \le 2\lambda^2 - 2\lambda \langle Tx_n, x_n \rangle \to 0.\] By compactness, a subsequence has \(Tx_{n_k} \to y\). Then \(\lambda x_{n_k} \to y\) as well, so \(x_{n_k} \to x = y / \lambda\), a unit vector, and continuity gives \(Tx = \lambda x\). \(\blacksquare\)

The spectral theorem

Theorem 11.6 (spectral theorem for compact self-adjoint operators; Hilbert 1906, Schmidt 1907). Let \(T\) be a compact self-adjoint operator on a Hilbert space \(H\). There is an orthonormal sequence \((e_k)\), finite or infinite, of eigenvectors with nonzero real eigenvalues \(\lambda_k\), such that \(|\lambda_1| \ge |\lambda_2| \ge \cdots\), \(\lambda_k \to 0\) if the sequence is infinite, and \[Tx = \sum_k \lambda_k \langle x, e_k \rangle e_k \quad \text{for every } x \in H.\] Moreover \(H = \ker T \oplus \overline{\operatorname{span}}\{e_k\}\). In particular, if \(\ker T = \{0\}\) and \(H\) is separable, the eigenvectors form an orthonormal basis.

Proof. Let \(H_1 = H\) and \(T_1 = T\). If \(T_1 \ne 0\), Theorem 11.5 gives a unit eigenvector \(e_1\) with eigenvalue \(\lambda_1\), \(|\lambda_1| = \|T\|\). Let \(H_2 = \{e_1\}^\perp\). It is invariant under \(T\): if \(x \perp e_1\) then \(\langle Tx, e_1 \rangle = \langle x, Te_1 \rangle = \lambda_1 \langle x, e_1 \rangle = 0\). The restriction \(T_2\) of \(T\) to \(H_2\) is compact and self-adjoint, with \(\|T_2\| \le \|T_1\|\). Repeat: as long as \(T_n \ne 0\), obtain \(e_n \in H_n\) with \(|\lambda_n| = \|T_n\|\), and set \(H_{n+1} = \{e_1, \ldots, e_n\}^\perp\). If some \(T_{n+1} = 0\), the process stops with finitely many eigenvectors. Otherwise it produces an orthonormal sequence, which converges weakly to \(0\), so \(Te_n = \lambda_n e_n \to 0\) in norm by Proposition 10.5, and \(|\lambda_n| \to 0\).

For the expansion, let \(x \in H\) and \(y_n = x - \sum_{k < n} \langle x, e_k \rangle e_k \in H_n\). Then \(\|Ty_n\| \le \|T_n\| \, \|y_n\| \le |\lambda_n| \, \|x\|\), which tends to zero, or vanishes once the process has stopped. Since \(Ty_n = Tx - \sum_{k < n} \lambda_k \langle x, e_k \rangle e_k\), the expansion follows. Finally, the expansion shows that \(Tx = 0\) exactly when \(\langle x, e_k \rangle = 0\) for all \(k\), since the \(\lambda_k\) are nonzero, so \(\ker T = \{e_k\}^\perp\), which is the orthogonal decomposition claimed. \(\blacksquare\)

Two consequences are worth stating. First, the variational characterization: \(\lambda_1 = \pm\max_{\|x\| = 1} |\langle Tx, x \rangle|\), and for a positive compact operator with eigenvalues \(\lambda_1 \ge \lambda_2 \ge \cdots\), the Courant–Fischer min–max principle \(\lambda_n = \min_{\dim V = n - 1} \max\{\langle Tx, x \rangle : \|x\| = 1, x \perp V\}\) describes each eigenvalue without reference to the others. Second, functions of \(T\) are defined by acting on the eigenvalues: \(f(T)x = \sum f(\lambda_k) \langle x, e_k \rangle e_k\) for bounded \(f\) with \(f(0) = 0\) (or add \(f(0)\) times the projection onto \(\ker T\)). For instance, \(I - \mu T\) is invertible exactly when \(\frac{1}{\mu}\) is not an eigenvalue, and then \((I - \mu T)^{-1} x = \sum \frac{1}{1 - \mu \lambda_k} \langle x, e_k \rangle e_k + P_{\ker T}x\).

The vibrating string

Consider the boundary value problem \(-u^{\prime\prime} = f\) on \([0, 1]\) with \(u(0) = u(1) = 0\). Integrating twice and imposing the boundary conditions gives the unique solution \[u(x) = \int_0^1 G(x, y) f(y) \, dy, \qquad G(x, y) = \min(x, y) - xy = \begin{cases} y(1 - x), & y \le x, \\ x(1 - y), & y \ge x. \end{cases}\] One checks this by differentiating \(u(x) = (1 - x)\int_0^x y f(y) \, dy + x \int_x^1 (1 - y) f(y) \, dy\) twice. The Green's operator \(G\) has a continuous, real, symmetric kernel, so it is compact and self-adjoint on \(L^2(0, 1)\), and it is injective: \(Gf = 0\) means the solution of \(-u^{\prime\prime} = f\) is zero, so \(f = 0\).

Its eigenvectors are found from \(G\phi = \mu \phi\), that is, \(-\mu\phi^{\prime\prime} = \phi\) with \(\phi(0) = \phi(1) = 0\). The solutions are \(\phi_n(x) = \sqrt{2} \sin n\pi x\) with \(\mu_n = \frac{1}{n^2 \pi^2}\), \(n = 1, 2, \ldots\). The spectral theorem, together with injectivity of \(G\), shows that \(\{\sqrt{2} \sin n\pi x\}\) is an orthonormal basis of \(L^2(0, 1)\), the Fourier sine basis, now obtained without any approximation theorem. The eigenvalues of the differential operator \(-\frac{d^2}{dx^2}\) are the reciprocals \(n^2\pi^2\), increasing to infinity: the unbounded operator has an inverse that is compact. For a string of length \(\ell\), tension \(\tau\) and linear density \(\rho\), the same analysis gives the natural frequencies \(\frac{n}{2\ell}\sqrt{\tau / \rho}\), the harmonic series of musical acoustics.

Mercer's theorem of 1909 says that for a continuous positive kernel the eigenfunction expansion of the kernel itself converges uniformly: \[\min(x, y) - xy = \sum_{n=1}^{\infty} \frac{2 \sin n\pi x \, \sin n\pi y}{n^2 \pi^2}.\] Setting \(x = y\) and integrating gives \(\int_0^1 (x - x^2) \, dx = \frac{1}{6} = \sum_n \frac{1}{n^2\pi^2}\), which is once more Euler's sum \(\sum 1/n^2 = \pi^2 / 6\): the trace of the Green's operator computed in two ways. More general Sturm–Liouville problems \(-(pu^{\prime})^{\prime} + qu = \lambda w u\), studied by Charles Sturm and Joseph Liouville in 1836 and 1837, are treated in exactly the same way, and they produce the Legendre polynomials, Bessel functions and Hermite functions of mathematical physics as eigenfunction bases.

# Eigenvalues of -u'' on (0,1) with u(0) = u(1) = 0: finite differences versus (n pi)^2,
# and the largest eigenvalues of the discretized Green's operator versus 1/(n pi)^2.
import math

N = 60                       # interior grid points
h = 1 / (N + 1)
# symmetric tridiagonal matrix (2, -1)/h^2; its eigenvalues are known in closed form,
# but we compute them with the cyclic Jacobi rotation method to keep the check honest
A = [[(2.0 if i == j else -1.0 if abs(i - j) == 1 else 0.0) / h ** 2 for j in range(N)] for i in range(N)]
for sweep in range(12):
    for p in range(N - 1):
        for q in range(p + 1, N):
            if abs(A[p][q]) < 1e-12:
                continue
            theta = (A[q][q] - A[p][p]) / (2 * A[p][q])
            t = (1 if theta >= 0 else -1) / (abs(theta) + math.sqrt(theta * theta + 1))
            c = 1 / math.sqrt(t * t + 1); s = t * c
            for k in range(N):
                akp, akq = A[k][p], A[k][q]
                A[k][p], A[k][q] = c * akp - s * akq, s * akp + c * akq
            for k in range(N):
                apk, aqk = A[p][k], A[q][k]
                A[p][k], A[q][k] = c * apk - s * aqk, s * apk + c * aqk
ev = sorted(A[i][i] for i in range(N))
for n in range(1, 6):
    exact_fd = 4 / h ** 2 * math.sin(n * math.pi * h / 2) ** 2
    print(f"n = {n}: finite-difference eigenvalue = {ev[n - 1]:9.4f}  closed form = {exact_fd:9.4f}  (n pi)^2 = {(n * math.pi) ** 2:9.4f}")

# Green's operator G(x, y) = min(x, y) - x y, Nystrom discretization, power iteration with deflation
xs = [(i + 1) * h for i in range(N)]
Gm = [[(min(a, b) - a * b) * h for b in xs] for a in xs]
vecs = []
for n in range(1, 4):
    v = [math.sin(7 * k + n) for k in range(N)]
    for _ in range(400):
        w = [sum(Gm[i][j] * v[j] for j in range(N)) for i in range(N)]
        for u in vecs:                                   # deflate earlier eigenvectors
            c = sum(a * b for a, b in zip(w, u)); w = [a - c * b for a, b in zip(w, u)]
        nrm = math.sqrt(sum(a * a for a in w)); v = [a / nrm for a in w]
    vecs.append(v)
    mu = sum(v[i] * sum(Gm[i][j] * v[j] for j in range(N)) for i in range(N))
    print(f"Green's operator: mu_{n} = {mu:.6f}   1/(n pi)^2 = {1 / (n * math.pi) ** 2:.6f}")
print("trace of the discretized Green's operator:", round(sum(Gm[i][i] for i in range(N)), 5), " exact 1/6 =", round(1 / 6, 5))

n = 1: finite-difference eigenvalue =    9.8674  closed form =    9.8674  (n pi)^2 =    9.8696
n = 2: finite-difference eigenvalue =   39.4435  closed form =   39.4435  (n pi)^2 =   39.4784
n = 3: finite-difference eigenvalue =   88.6499  closed form =   88.6499  (n pi)^2 =   88.8264
n = 4: finite-difference eigenvalue =  157.3560  closed form =  157.3560  (n pi)^2 =  157.9137
n = 5: finite-difference eigenvalue =  245.3797  closed form =  245.3797  (n pi)^2 =  246.7401
Green's operator: mu_1 = 0.101344   1/(n pi)^2 = 0.101321
Green's operator: mu_2 = 0.025353   1/(n pi)^2 = 0.025330
Green's operator: mu_3 = 0.011280   1/(n pi)^2 = 0.011258
trace of the discretized Green's operator: 0.16662  exact 1/6 = 0.16667

The output shows the low eigenvalues of the discrete problem agreeing with \((n\pi)^2\) with relative error below \(0.006\) for \(n \le 5\), the error growing with \(n\) because the grid resolves fewer points per wavelength. The closed form \(\frac{4}{h^2}\sin^2\frac{n\pi h}{2}\) is exact for the matrix, and it tends to \((n\pi)^2\) as \(h \to 0\). The eigenvalues of the discretized Green's operator agree with \(\frac{1}{n^2\pi^2}\), and the discrete trace approximates \(\frac{1}{6}\).