Normed and Banach spaces

Completeness is the property that makes analysis work. In a complete space every Cauchy sequence converges, so a limit can be shown to exist without knowing it in advance; that is how we solve equations by iteration, sum infinite series and construct solutions of differential equations. This lesson defines Banach spaces, proves that the main sequence and function spaces are complete, shows how completeness fails for an integral norm on continuous functions, gives a series criterion for completeness, and proves Riesz's lemma, which explains once and for all why the closed unit ball of an infinite-dimensional normed space is never compact.

Banach spaces

Definition 2.1. A Banach space is a normed space that is complete in the metric \(d(x, y) = \|x - y\|\). The name honors Stefan Banach, whose 1922 thesis, published in Fundamenta Mathematicae, began the systematic study of complete normed spaces; he called them spaces of type (B).

The main examples, all of which we use repeatedly, are the following. (a) \(\mathbb{K}^n\) with any norm, complete by Theorem 1.3. (b) For \(1 \le p < \infty\), the sequence space \(\ell^p\) of all \(x = (x_1, x_2, \ldots)\) with \(\|x\|_p = (\sum_n |x_n|^p)^{1/p} < \infty\). (c) The space \(\ell^\infty\) of bounded sequences with \(\|x\|_\infty = \sup_n |x_n|\), and its closed subspaces \(c\) of convergent sequences and \(c_0\) of sequences tending to zero. (d) The space \(C(K)\) of continuous functions on a compact metric space \(K\) with \(\|f\|_\infty = \max_K |f|\); the leading case is \(C[a, b]\). (e) The spaces \(L^p(\mu)\) of MATH-441, complete by the Riesz–Fischer theorem. (f) The space \(C^1[a, b]\) of continuously differentiable functions with \(\|f\|_{C^1} = \|f\|_\infty + \|f^{\prime}\|_\infty\).

Sequence spaces are the special case of \(L^p(\mu)\) for counting measure on \(\mathbb{N}\), so their completeness follows from MATH-441. Because the argument is the template for all completeness proofs in this course, we give it directly.

Theorem 2.2. For \(1 \le p \le \infty\), the space \(\ell^p\) is a Banach space.

Proof. Minkowski's inequality makes \(\|\cdot\|_p\) a norm. Let \((x^{(k)})_{k \ge 1}\) be a Cauchy sequence in \(\ell^p\), where \(x^{(k)} = (x^{(k)}_1, x^{(k)}_2, \ldots)\). Step 1, a candidate limit: for each coordinate \(n\), \(|x^{(k)}_n - x^{(j)}_n| \le \|x^{(k)} - x^{(j)}\|_p\), so \((x^{(k)}_n)_k\) is a Cauchy sequence of scalars and converges to some \(x_n\). Let \(x = (x_n)\). Step 2, the limit is in the space and the convergence is in norm: fix \(\varepsilon > 0\) and choose \(K\) with \(\|x^{(k)} - x^{(j)}\|_p < \varepsilon\) for \(j, k \ge K\). For \(p < \infty\) and any \(N\), \[\sum_{n=1}^{N} |x^{(k)}_n - x^{(j)}_n|^p < \varepsilon^p.\] Let \(j \to \infty\) in this finite sum: \(\sum_{n=1}^{N} |x^{(k)}_n - x_n|^p \le \varepsilon^p\). Now let \(N \to \infty\): \(\|x^{(k)} - x\|_p \le \varepsilon\) for \(k \ge K\). In particular \(x^{(K)} - x \in \ell^p\), so \(x = x^{(K)} - (x^{(K)} - x) \in \ell^p\), and \(x^{(k)} \to x\) in \(\ell^p\). For \(p = \infty\) the same argument works with the finite sum replaced by \(\max_{n \le N} |x^{(k)}_n - x_n| \le \varepsilon\). \(\blacksquare\)

The proof has a pattern worth naming. First use a weaker kind of convergence (here coordinatewise) to produce a candidate limit. Then pass to the limit in an inequality restricted to finitely many terms, and only afterwards remove the restriction. Passing to the limit before truncating would require exchanging a limit with an infinite sum, which is exactly what we cannot yet justify.

Theorem 2.3. The space \(C[a, b]\) with the supremum norm is a Banach space.

Proof. Let \((f_k)\) be Cauchy in \(\|\cdot\|_\infty\). For each \(x\), \(|f_k(x) - f_j(x)| \le \|f_k - f_j\|_\infty\), so \(f_k(x)\) converges to a number \(f(x)\). Given \(\varepsilon > 0\), choose \(K\) with \(\|f_k - f_j\|_\infty < \varepsilon\) for \(j, k \ge K\). Letting \(j \to \infty\) at each fixed \(x\) gives \(|f_k(x) - f(x)| \le \varepsilon\) for all \(x\) and \(k \ge K\), so \(f_k \to f\) uniformly. It remains to show \(f\) is continuous. Fix \(x_0\) and \(k \ge K\), and choose \(\delta > 0\) with \(|f_k(x) - f_k(x_0)| < \varepsilon\) when \(|x - x_0| < \delta\). For such \(x\), \[|f(x) - f(x_0)| \le |f(x) - f_k(x)| + |f_k(x) - f_k(x_0)| + |f_k(x_0) - f(x_0)| < 3\varepsilon.\] So \(f \in C[a, b]\) and \(\|f_k - f\|_\infty \le \varepsilon\) for \(k \ge K\). \(\blacksquare\) The same proof works for \(C(K)\) on any compact metric space, and for bounded continuous functions on any metric space.

The space \(c_0\) is closed in \(\ell^\infty\): if \(x^{(k)} \in c_0\) and \(\|x^{(k)} - x\|_\infty < \varepsilon\), then \(|x_n| < \varepsilon + |x^{(k)}_n|\), which is less than \(2\varepsilon\) for large \(n\). A closed subspace of a Banach space is itself a Banach space, since a Cauchy sequence in the subspace converges in the big space and its limit stays in the subspace. So \(c_0\) and \(c\) are Banach spaces. The space \(c_{00}\) of sequences with only finitely many nonzero terms is not closed in any \(\ell^p\), \(p < \infty\), nor in \(c_0\): the truncations of \((1, \frac{1}{2}, \frac{1}{3}, \ldots)\) lie in \(c_{00}\) and converge in \(\ell^2\) and \(c_0\) to a sequence outside it.

A Cauchy sequence with no continuous limit

Completeness depends on the norm, not only on the vector space. On \(C[0, 1]\) use the integral norm \(\|f\|_1 = \int_0^1 |f|\). For \(n \ge 2\) let \(f_n\) be the continuous function equal to \(0\) on \([0, \frac{1}{2} - \frac{1}{n}]\), equal to \(1\) on \([\frac{1}{2} + \frac{1}{n}, 1]\) and linear in between.

For \(m > n\), the functions \(f_m\) and \(f_n\) differ only on \([\frac{1}{2} - \frac{1}{n}, \frac{1}{2} + \frac{1}{n}]\), where both lie in \([0, 1]\), so \(\|f_m - f_n\|_1 \le \frac{2}{n}\). Thus \((f_n)\) is Cauchy in \(\|\cdot\|_1\). Suppose some continuous \(f\) satisfied \(\|f_n - f\|_1 \to 0\). On \([0, \frac{1}{2} - \delta]\), for \(n > \frac{1}{\delta}\), \(f_n = 0\), so \(\int_0^{1/2 - \delta} |f| \le \|f_n - f\|_1 \to 0\), and continuity forces \(f = 0\) on \([0, \frac{1}{2})\). Similarly \(f = 1\) on \((\frac{1}{2}, 1]\). No continuous function does both, so \((C[0, 1], \|\cdot\|_1)\) is not complete. Its completion, the smallest complete space containing it as a dense subspace, is \(L^1[0, 1]\); this is one reason the Lebesgue integral of MATH-441 is the right tool.

Every normed space has a completion, unique up to an isometric isomorphism. One construction takes equivalence classes of Cauchy sequences, as in the construction of the real numbers from the rationals. A slicker construction, using the dual space and the Hahn–Banach theorem, appears in lesson 7.

Absolutely convergent series

In \(\mathbb{R}\), a series \(\sum x_k\) with \(\sum |x_k| < \infty\) converges. The same statement in a normed space characterizes completeness.

Theorem 2.4. A normed space \(X\) is complete if and only if every absolutely convergent series converges: whenever \(\sum_k \|x_k\| < \infty\), the partial sums \(s_n = \sum_{k=1}^{n} x_k\) converge in \(X\).

Proof. Suppose \(X\) is complete and \(\sum \|x_k\| < \infty\). For \(m > n\), \(\|s_m - s_n\| \le \sum_{k=n+1}^{m} \|x_k\|\), which is small for large \(n\), so \((s_n)\) is Cauchy and converges. Conversely suppose absolutely convergent series converge, and let \((y_n)\) be Cauchy. Choose \(n_1 < n_2 < \cdots\) with \(\|y_m - y_n\| < 2^{-k}\) for \(m, n \ge n_k\). The series \(y_{n_1} + \sum_{k \ge 1} (y_{n_{k+1}} - y_{n_k})\) is absolutely convergent, so it converges; its partial sums are \(y_{n_{k+1}}\), so the subsequence \((y_{n_k})\) converges to some \(y\). A Cauchy sequence with a convergent subsequence converges to the same limit: \(\|y_m - y\| \le \|y_m - y_{n_k}\| + \|y_{n_k} - y\|\). \(\blacksquare\)

This criterion is how MATH-441 proved that \(L^p\) is complete, and it will prove completeness of operator spaces in lesson 3 and justify the Neumann series. It also explains why \(c_{00}\) fails: \(\sum_k 2^{-k} e_k\) is absolutely convergent in \(\ell^2\) with all partial sums in \(c_{00}\), but its limit has infinitely many nonzero entries.

Subspaces, quotients and separability

If \(M\) is a closed subspace of a normed space \(X\), the quotient space \(X / M\) of cosets \(x + M\) carries the quotient norm \(\|x + M\| = \inf_{m \in M} \|x - m\| = \operatorname{dist}(x, M)\). Closedness of \(M\) ensures (N1): if the distance is zero, then \(x\) is a limit of elements of \(M\), so \(x \in M\) and \(x + M\) is the zero coset. If \(X\) is a Banach space, so is \(X / M\): an absolutely convergent series \(\sum (x_k + M)\) can be lifted to representatives with \(\|x_k\| < \|x_k + M\| + 2^{-k}\), whose series converges in \(X\), and the image of the sum is the sum of the cosets. Quotients appear in the open mapping theorem of lesson 8 and in Fredholm theory.

A normed space is separable if it has a countable dense subset. The spaces \(\ell^p\) for \(p < \infty\) and \(c_0\) are separable: finitely supported sequences with rational entries are dense. The space \(C[a, b]\) is separable, because polynomials with rational coefficients are dense by the Weierstrass approximation theorem. The space \(\ell^\infty\) is not separable. Proof: for each subset \(A \subseteq \mathbb{N}\) the indicator sequence \(\mathbf{1}_A\) lies in \(\ell^\infty\), and \(\|\mathbf{1}_A - \mathbf{1}_B\|_\infty = 1\) for \(A \ne B\). The balls of radius \(\frac{1}{2}\) around these points are disjoint, and there are uncountably many of them, so any dense set, which must meet every ball, is uncountable. \(\blacksquare\) The same argument shows that \(L^\infty[0, 1]\) is not separable.

Riesz's lemma and the failure of compactness

Lemma 2.5 (Riesz's lemma, 1918). Let \(Y\) be a closed proper subspace of a normed space \(X\), and let \(0 < \theta < 1\). Then there is \(x \in X\) with \(\|x\| = 1\) and \(\operatorname{dist}(x, Y) \ge \theta\).

Proof. Choose \(z \in X \setminus Y\). Since \(Y\) is closed, \(d = \operatorname{dist}(z, Y) > 0\). Since \(d / \theta > d\), there is \(y_0 \in Y\) with \(\|z - y_0\| \le d / \theta\). Let \(x = (z - y_0) / \|z - y_0\|\), a unit vector. For any \(y \in Y\), the vector \(y_0 + \|z - y_0\| y\) lies in \(Y\), so \[\|x - y\| = \frac{\|z - (y_0 + \|z - y_0\| \, y)\|}{\|z - y_0\|} \ge \frac{d}{d / \theta} = \theta. \qquad \blacksquare\]

Theorem 2.6. The closed unit ball of a normed space \(X\) is compact if and only if \(X\) is finite dimensional.

Proof. If \(\dim X < \infty\), the ball is closed and bounded, hence compact by Theorem 1.3 and Heine–Borel. Suppose \(\dim X = \infty\). Choose a unit vector \(x_1\). Having chosen unit vectors \(x_1, \ldots, x_n\), their span \(Y_n\) is finite dimensional, hence closed, and proper. Riesz's lemma with \(\theta = \frac{1}{2}\) gives a unit vector \(x_{n+1}\) with \(\operatorname{dist}(x_{n+1}, Y_n) \ge \frac{1}{2}\). Then \(\|x_n - x_m\| \ge \frac{1}{2}\) for all \(n \ne m\), so \((x_n)\) has no Cauchy subsequence and the unit ball is not sequentially compact. \(\blacksquare\)

In \(\ell^2\) the lemma holds even with \(\theta = 1\), as the unit vectors \(e_n\) show, and lesson 4 proves this for every Hilbert space using orthogonal projections. In general Banach spaces \(\theta = 1\) can fail. In the space \(X\) of functions in \(C[0, 1]\) that vanish at \(0\), with the supremum norm, the closed subspace \(Y\) of functions with \(\int_0^1 f = 0\) has no unit vector of \(X\) at distance exactly \(1\) from it.

# The ramps f_n on [0, 1]: Cauchy in the integral norm, not in the sup norm.
def ramp(n):
    a, b = 0.5 - 1 / n, 0.5 + 1 / n
    return lambda x: 0.0 if x <= a else 1.0 if x >= b else (x - a) / (b - a)

M = 200000
xs = [(i + 0.5) / M for i in range(M)]
def dist1(f, g):
    return sum(abs(f(x) - g(x)) for x in xs) / M
def distsup(f, g):
    return max(abs(f(x) - g(x)) for x in xs)

for n, m in ((4, 8), (16, 32), (64, 128), (256, 512)):
    f, g = ramp(n), ramp(m)
    print(f"n = {n:3d}, m = {m:3d}:  integral distance = {dist1(f, g):.5f}   sup distance = {distsup(f, g):.3f}")

step = lambda x: 0.0 if x < 0.5 else 1.0
print("integral distance from f_256 to the step function:", round(dist1(ramp(256), step), 5))
# the exact integral distance between f_n and f_m (m > n) is 1/(2n) - 1/(2m)
print("exact value for n = 4, m = 8:", 1 / 8 - 1 / 16)

n =   4, m =   8:  integral distance = 0.06250   sup distance = 0.250
n =  16, m =  32:  integral distance = 0.01562   sup distance = 0.250
n =  64, m = 128:  integral distance = 0.00391   sup distance = 0.250
n = 256, m = 512:  integral distance = 0.00098   sup distance = 0.250
integral distance from f_256 to the step function: 0.00195
exact value for n = 4, m = 8: 0.0625

The output confirms that \(\|f_n - f_m\|_1 = \frac{1}{2n} - \frac{1}{2m}\) for \(m > n\), which a short computation of the area between the two ramps gives exactly, while the supremum distance between \(f_n\) and \(f_{2n}\) is \(\frac{1}{4}\) for every \(n\), attained at \(x = \frac{1}{2} \pm \frac{1}{2n}\). So the ramps are not Cauchy in the norm in which \(C[0, 1]\) is complete, which is consistent with Theorem 2.3.