Weak topologies and reflexivity

The closed unit ball of an infinite-dimensional normed space is never compact in the norm topology. Since compactness is the main tool for proving that minimizers, limits and solutions exist, we need a substitute. The idea is to weaken the notion of convergence: instead of asking that \(\|x_n - x\| \to 0\), ask only that every continuous linear functional sees \(x_n\) approach \(x\). With this weak convergence, bounded sequences in Hilbert spaces and in the other reflexive spaces always have convergent subsequences. This lesson develops weak and weak-star convergence, proves the Banach–Alaoglu theorem in its sequential form, characterizes reflexive spaces, and applies weak compactness to show that convex minimization problems have solutions.

Weak convergence

Definition 9.1. A sequence \((x_n)\) in a normed space \(X\) converges weakly to \(x\), written \(x_n \overset{w}{\longrightarrow} x\), if \(f(x_n) \to f(x)\) for every \(f \in X^*\). Weak limits are unique, because \(X^*\) separates points (Corollary 7.6), and norm convergence implies weak convergence, since \(|f(x_n) - f(x)| \le \|f\| \, \|x_n - x\|\). In a Hilbert space, by the Riesz theorem, \(x_n \overset{w}{\longrightarrow} x\) means \(\langle x_n, y \rangle \to \langle x, y \rangle\) for every \(y\).

Example 9.2. Every orthonormal sequence \((e_n)\) in a Hilbert space converges weakly to \(0\): by Bessel's inequality, \(\sum_n |\langle y, e_n \rangle|^2 \le \|y\|^2\), so \(\langle e_n, y \rangle \to 0\) for every \(y\). Yet \(\|e_n\| = 1\), so \(e_n\) does not converge in norm. In \(L^2(0, \pi)\), the functions \(\sin nx\) are orthogonal with \(\|\sin nx\|^2 = \frac{\pi}{2}\), so \(\sin nx \overset{w}{\longrightarrow} 0\): integrated against any fixed square-integrable \(g\), the faster and faster oscillations cancel. This is the \(L^2\) form of the Riemann–Lebesgue lemma.

On \(\ell^p\) with \(1 < p < \infty\), the unit vectors satisfy \(e_n \overset{w}{\longrightarrow} 0\), since every functional is \(x \mapsto \sum y_k x_k\) with \(y \in \ell^q\), and \(y_n \to 0\). The same holds in \(c_0\), where \(y \in \ell^1\). In \(\ell^1\) the unit vectors do not converge weakly to \(0\): the functional given by \(y = (1, 1, 1, \ldots) \in \ell^\infty\) takes the value \(1\) on each. In fact Schur proved in 1921 that weakly convergent sequences in \(\ell^1\) converge in norm, so \(\ell^1\) has no interesting weakly convergent sequences at all, although its weak topology still differs from its norm topology.

Proposition 9.3. If \(x_n \overset{w}{\longrightarrow} x\) in a normed space, then \((x_n)\) is bounded and \(\|x\| \le \liminf \|x_n\|\). Proof. The functionals \(Jx_n\) on the Banach space \(X^*\) satisfy \(\sup_n |Jx_n(f)| = \sup_n |f(x_n)| < \infty\) for each \(f\), since convergent sequences of scalars are bounded. By the uniform boundedness principle, \(\sup_n \|x_n\| = \sup_n \|Jx_n\| < \infty\). For the second claim choose \(f\) with \(\|f\| = 1\) and \(f(x) = \|x\|\); then \(\|x\| = \lim |f(x_n)| \le \liminf \|x_n\|\). \(\blacksquare\)

The norm can drop in the limit, as \(e_n \overset{w}{\longrightarrow} 0\) shows, but it cannot jump up: the norm is weakly lower semicontinuous. In a Hilbert space, the drop is the only thing that separates weak from norm convergence. Proposition 9.4. In a Hilbert space, if \(x_n \overset{w}{\longrightarrow} x\) and \(\|x_n\| \to \|x\|\), then \(x_n \to x\) in norm. Proof: \(\|x_n - x\|^2 = \|x_n\|^2 - 2 \operatorname{Re} \langle x_n, x \rangle + \|x\|^2 \to \|x\|^2 - 2\|x\|^2 + \|x\|^2 = 0\). \(\blacksquare\)

The weak topology

Weak convergence is convergence in a topology, the weak topology \(\sigma(X, X^*)\), the coarsest topology making every \(f \in X^*\) continuous. Its basic open neighborhoods of a point \(x_0\) are \[U(x_0; f_1, \ldots, f_k; \varepsilon) = \{x : |f_i(x) - f_i(x_0)| < \varepsilon, \ i = 1, \ldots, k\},\] determined by finitely many functionals. In infinite dimensions every such neighborhood contains the affine subspace \(x_0 + \bigcap_i \ker f_i\), which is infinite dimensional. So weak open sets are unbounded, the open unit ball has empty weak interior, and the weak topology on an infinite-dimensional space is strictly coarser than the norm topology and not metrizable. In finite dimensions the two topologies agree.

Despite this, closed convex sets are the same in both topologies. Theorem 9.5 (Mazur, 1933). A convex subset of a normed space is weakly closed if and only if it is norm closed. In particular, if \(x_n \overset{w}{\longrightarrow} x\), then some sequence of convex combinations of the \(x_n\) converges to \(x\) in norm. Proof. Weakly closed sets are norm closed because the weak topology is coarser. Conversely, let \(C\) be norm closed and convex and \(x_0 \notin C\). Some ball \(B(x_0, r)\) misses \(C\); by Theorem 7.9 applied to the open convex set \(B(x_0, r)\) and the convex set \(C\), there are a functional \(F\) and \(\gamma\) with \(\operatorname{Re} F < \gamma \le \operatorname{Re} F\) on the ball and on \(C\) respectively. Then \(\{x : \operatorname{Re} F(x) < \gamma\}\) is a weak neighborhood of \(x_0\) disjoint from \(C\). For the second statement, the norm closure of the convex hull of \(\{x_n : n \ge N\}\) is weakly closed and contains the weak limit \(x\), for every \(N\). \(\blacksquare\)

For the unit vectors of \(\ell^2\), the averages \(\frac{1}{n}(e_1 + \cdots + e_n)\) have norm \(\frac{1}{\sqrt{n}} \to 0\): the convex combinations converge in norm to the weak limit, as Mazur predicts.

Weak-star convergence and the Banach–Alaoglu theorem

On a dual space \(X^*\) there is an even weaker topology. A sequence \(f_n \in X^*\) converges weak-star to \(f\), written \(f_n \overset{w*}{\longrightarrow} f\), if \(f_n(x) \to f(x)\) for every \(x \in X\). This is convergence against the elements of \(J(X) \subseteq X^{**}\) only, rather than against all of \(X^{**}\). By the uniform boundedness principle, applied on the Banach space \(X\), weak-star convergent sequences in the dual of a Banach space are bounded.

Example 9.6. In \(C[0, 1]^*\), the point evaluations \(\delta_{1/n}\) converge weak-star to \(\delta_0\), since \(f(1/n) \to f(0)\) for continuous \(f\). They do not converge in norm, because \(\|\delta_a - \delta_b\| = 2\) for \(a \ne b\), as a continuous function with values \(1\) at \(a\) and \(-1\) at \(b\) shows. Weak-star convergence of probability measures is the convergence in distribution of probability theory, and the central limit theorem is a statement about it.

Theorem 9.7 (Banach–Alaoglu, sequential form). If \(X\) is a separable normed space, every bounded sequence in \(X^*\) has a weak-star convergent subsequence.

Proof. Let \(\|f_n\| \le C\) and let \(\{x_1, x_2, \ldots\}\) be dense in \(X\). The scalars \(f_n(x_1)\) are bounded, so some subsequence \(f_n^{(1)}\) has \(f_n^{(1)}(x_1)\) convergent. From it extract a subsequence \(f_n^{(2)}\) with \(f_n^{(2)}(x_2)\) convergent, and so on. The diagonal sequence \(g_n = f_n^{(n)}\) is eventually a subsequence of each \(f^{(k)}\), so \(g_n(x_k)\) converges for every \(k\). For arbitrary \(x\) and \(\varepsilon > 0\), choose \(x_k\) with \(\|x - x_k\| < \varepsilon\); then \(|g_n(x) - g_m(x)| \le 2C\varepsilon + |g_n(x_k) - g_m(x_k)|\), so \(g_n(x)\) is Cauchy and converges to a limit \(g(x)\). The limit is linear with \(|g(x)| \le C\|x\|\), so \(g \in X^*\) and \(g_n \overset{w*}{\longrightarrow} g\). \(\blacksquare\)

Banach proved this sequential form in his book of 1932. Leonidas Alaoglu proved in 1940 the general form, without separability: the closed unit ball of \(X^*\) is compact in the weak-star topology. The proof embeds the ball into the product \(\prod_{x \in X} \{z \in \mathbb{K} : |z| \le \|x\|\}\) of compact discs, which is compact by Tychonoff's theorem, and checks that the image is closed, since pointwise limits of linear functionals of norm at most one are again such functionals. Without separability the compactness need not be sequential: the unit ball of \((\ell^\infty)^*\) contains a sequence with no weak-star convergent subsequence.

Reflexive spaces

Definition 9.8. A normed space \(X\) is reflexive if the canonical embedding \(J : X \to X^{**}\) is onto. A reflexive space is complete, being isometric to the dual space \(X^{**}\). On a reflexive space weak and weak-star convergence in \(X^*\) coincide, and the weak topology of \(X\) is the weak-star topology of \(X^{**}\) transported by \(J\).

Examples. Finite-dimensional spaces are reflexive by dimension count. Hilbert spaces are reflexive: given \(\Phi \in H^{**}\), the map \(y \mapsto \overline{\Phi(f_y)}\) is linear, because the conjugation undoes the conjugate linearity of \(y \mapsto f_y\), and bounded, so by Riesz it equals \(y \mapsto \langle y, x \rangle\) for some \(x\); then \(\Phi(f_y) = \langle x, y \rangle = f_y(x) = Jx(f_y)\), and every element of \(H^*\) is some \(f_y\). For \(1 < p < \infty\), applying Theorem 3.8 twice identifies \((\ell^p)^{**}\) with \(\ell^p\), and one checks that the identification is \(J\); the same holds for \(L^p(\mu)\). The spaces \(c_0\), \(\ell^1\), \(\ell^\infty\), \(C[0, 1]\) and \(L^1[0, 1]\) are not reflexive. For \(c_0\): \(c_0^* = \ell^1\) and \(c_0^{**} = (\ell^1)^* = \ell^\infty\), and \(J\) is the inclusion of \(c_0\) in \(\ell^\infty\), which is not onto. For \(\ell^1\): if a separable space \(X\) is reflexive, then \(X^{**} = J(X)\) is separable, so \(X^*\) is separable by Corollary 7.8 applied to \(X^*\). But \(\ell^1\) is separable and its dual \(\ell^\infty\) is not.

Theorem 9.9. In a Hilbert space, every bounded sequence has a weakly convergent subsequence. Proof. Let \(\|x_n\| \le C\), and let \(M\) be the closed linear span of the \(x_n\), a separable Hilbert space. The functionals \(f_n(y) = \langle y, x_n \rangle\) on \(M\) are bounded by \(C\), so by Theorem 9.7 a subsequence satisfies \(\langle y, x_{n_k} \rangle \to \phi(y)\) for every \(y \in M\), with \(\phi\) bounded and linear. By Riesz on \(M\), \(\phi(y) = \langle y, x \rangle\) for some \(x \in M\). For \(y \in M^\perp\), \(\langle y, x_{n_k} \rangle = 0 = \langle y, x \rangle\). Hence \(\langle y, x_{n_k} \rangle \to \langle y, x \rangle\) for all \(y \in H\), that is, \(x_{n_k} \overset{w}{\longrightarrow} x\). \(\blacksquare\)

The same statement holds in every reflexive Banach space; this is the Eberlein–Šmulian theorem, and conversely a Banach space in which every bounded sequence has a weakly convergent subsequence is reflexive. Another characterization is James's theorem of 1964: a Banach space is reflexive if and only if every continuous linear functional attains its norm on the closed unit ball. On \(c_0\) the functional given by \(y = (\frac{1}{2}, \frac{1}{4}, \frac{1}{8}, \ldots) \in \ell^1\) has norm \(1\) but \(|\sum y_n x_n| < 1\) whenever \(\|x\|_\infty \le 1\), since \(x \in c_0\) forces \(|x_n| < 1\) for large \(n\). Uniformly convex spaces, such as \(L^p\) for \(1 < p < \infty\), are reflexive by the Milman–Pettis theorem of 1938 and 1939.

Weak compactness and minimization

Theorem 9.10 (direct method). Let \(C\) be a nonempty closed convex subset of a Hilbert space, or more generally of a reflexive Banach space, and let \(\Phi : C \to \mathbb{R}\) be convex, norm continuous and coercive, meaning \(\Phi(x) \to \infty\) as \(\|x\| \to \infty\) in \(C\). Then \(\Phi\) attains its minimum on \(C\).

Proof (Hilbert case). Let \(m = \inf_C \Phi\) and choose \(x_n \in C\) with \(\Phi(x_n) \to m\). Coercivity makes \((x_n)\) bounded, so by Theorem 9.9 a subsequence converges weakly to some \(x\), which lies in \(C\) because \(C\) is weakly closed by Mazur's theorem. For every \(t > m\) the sublevel set \(\{\Phi \le t\}\) is closed and convex, hence weakly closed, and it contains \(x_{n_k}\) for large \(k\), so it contains \(x\). Thus \(\Phi(x) \le t\) for all \(t > m\), and \(\Phi(x) = m\), which in particular shows \(m > -\infty\). \(\blacksquare\)

This is the abstract skeleton of the direct method of the calculus of variations, introduced by Hilbert around 1900 to rescue Dirichlet's principle: minimize the energy \(\int |\nabla u|^2\) among functions with prescribed boundary values. The difficulty Weierstrass had pointed out, that a minimizing sequence might have no limit, disappears once the problem is posed in a Sobolev space, where bounded sequences have weakly convergent subsequences and the energy is weakly lower semicontinuous. Lesson 12 carries this out for the Dirichlet problem.

# Weak but not strong convergence in L^2(0, pi) and l^2, and weak-star convergence of point masses.
import math

M = 20000
h = math.pi / M
xs = [(k + 0.5) * h for k in range(M)]
g = [x * (math.pi - x) for x in xs]                       # a fixed test function
for n in (1, 3, 9, 33, 129):
    s = [math.sin(n * x) for x in xs]
    ip = h * sum(a * b for a, b in zip(g, s))
    nrm = math.sqrt(h * sum(b * b for b in s))
    exact = 0 if n % 2 == 0 else 4 / n ** 3                 # integral of x(pi - x) sin(n x)
    print(f"n = {n:3d}: <g, sin nx> = {ip: .6f} (exact {exact:.6f}),  |sin nx| = {nrm:.5f}")
print("sqrt(pi/2) =", round(math.sqrt(math.pi / 2), 5))

# Mazur: averages of e_1, ..., e_n in l^2 have norm 1/sqrt(n)
for n in (1, 4, 100, 10000):
    print(f"| (e_1 + ... + e_n)/n | for n = {n:5d}: {math.sqrt(n * (1 / n) ** 2):.4f}")

# weak-star: delta_{1/n}(f) = f(1/n) -> f(0) for continuous f, yet |delta_a - delta_b| = 2
f = lambda t: math.cos(3 * t) + t * t
print("f(1/n) for n = 1, 10, 1000:", [round(f(1 / n), 6) for n in (1, 10, 1000)], " f(0) =", f(0))

n =   1: <g, sin nx> =  4.000000 (exact 4.000000),  |sin nx| = 1.25331
n =   3: <g, sin nx> =  0.148148 (exact 0.148148),  |sin nx| = 1.25331
n =   9: <g, sin nx> =  0.005487 (exact 0.005487),  |sin nx| = 1.25331
n =  33: <g, sin nx> =  0.000111 (exact 0.000111),  |sin nx| = 1.25331
n = 129: <g, sin nx> =  0.000002 (exact 0.000002),  |sin nx| = 1.25331
sqrt(pi/2) = 1.25331
| (e_1 + ... + e_n)/n | for n =     1: 1.0000
| (e_1 + ... + e_n)/n | for n =     4: 0.5000
| (e_1 + ... + e_n)/n | for n =   100: 0.1000
| (e_1 + ... + e_n)/n | for n = 10000: 0.0100
f(1/n) for n = 1, 10, 1000: [0.010008, 0.965336, 0.999997]  f(0) = 1.0

The exact values follow from integrating by parts twice: \(\int_0^\pi x(\pi - x) \sin nx \, dx = \frac{2(1 - (-1)^n)}{n^3}\). Smoother test functions give faster decay of the coefficients, which is the quantitative content of weak convergence of \(\sin nx\) against regular functions; against a general \(g \in L^2\) the decay can be arbitrarily slow, but it always happens.