Cauchy sequences and series of real numbers

The monotone convergence theorem proves convergence without knowing the limit, but only for monotone sequences. Cauchy found a criterion that works for every sequence: a sequence converges exactly when its terms eventually get close to each other. This condition refers only to the terms, never to the limit, which makes it ideal for proving that something converges when we have no idea what it converges to. The main application is to infinite series. This lesson proves the Cauchy criterion, defines the sum of a series, and develops the comparison, ratio and root tests and the alternating series test, ending with absolute and conditional convergence and Riemann's rearrangement theorem.

Cauchy sequences

Definition. A sequence \((a_n)\) is a Cauchy sequence if for every \(\varepsilon > 0\) there is \(N\) such that \(|a_m - a_n| < \varepsilon\) for all \(m, n \ge N\).

Compare this with convergence. Convergence says the terms get close to a fixed number \(L\). The Cauchy condition says the terms get close to each other. Cauchy introduced the idea in his Cours d'analyse of 1821, although he did not prove the converse direction of the theorem below with full rigour, since the real numbers had not yet been constructed.

Lemma. Every convergent sequence is Cauchy.

Proof. Let \(a_n \to L\) and \(\varepsilon > 0\). Choose \(N\) with \(|a_n - L| < \frac{\varepsilon}{2}\) for \(n \ge N\). For \(m, n \ge N\), \(|a_m - a_n| \le |a_m - L| + |L - a_n| < \varepsilon\). \(\blacksquare\)

Lemma. Every Cauchy sequence is bounded.

Proof. With \(\varepsilon = 1\), choose \(N\) with \(|a_m - a_n| < 1\) for \(m, n \ge N\). Then \(|a_n| < |a_N| + 1\) for \(n \ge N\), and \(M = \max(|a_1|, \dots, |a_{N-1}|, |a_N| + 1)\) bounds the sequence. \(\blacksquare\)

Theorem (Cauchy criterion). A sequence of real numbers converges if and only if it is a Cauchy sequence.

Proof. We have shown convergent implies Cauchy. Conversely let \((a_n)\) be Cauchy. It is bounded, so by Bolzano–Weierstrass it has a subsequence \(a_{n_k} \to L\). We show the whole sequence converges to \(L\). Let \(\varepsilon > 0\). Choose \(N\) with \(|a_m - a_n| < \frac{\varepsilon}{2}\) for \(m, n \ge N\), and choose \(k\) with \(n_k \ge N\) and \(|a_{n_k} - L| < \frac{\varepsilon}{2}\). For \(n \ge N\), \(|a_n - L| \le |a_n - a_{n_k}| + |a_{n_k} - L| < \varepsilon\). \(\blacksquare\)

This theorem is a third form of completeness. In \(\mathbb{Q}\) the decimal truncations of \(\sqrt{2}\) are Cauchy but have no rational limit. Spaces in which every Cauchy sequence converges are called complete, and the idea extends far beyond the reals: MATH-341 and MATH-445 Functional Analysis study complete spaces of functions.

Be careful: it is not enough that consecutive terms get close. The partial sums \(h_n = 1 + \frac{1}{2} + \dots + \frac{1}{n}\) satisfy \(h_{n+1} - h_n = \frac{1}{n+1} \to 0\), yet the sequence is not Cauchy, as we now show.

Series

Definition. Given a sequence \((a_k)\), the series \(\sum_{k=1}^{\infty} a_k\) is the sequence of partial sums \(s_n = a_1 + a_2 + \dots + a_n\). The series converges to \(S\), written \(\sum_{k=1}^{\infty} a_k = S\), if \(s_n \to S\). Otherwise it diverges.

So a series is not a new kind of object: it is a sequence in disguise, and everything we know about sequences applies. The Cauchy criterion becomes:

Theorem (Cauchy criterion for series). \(\sum a_k\) converges if and only if for every \(\varepsilon > 0\) there is \(N\) with \(|a_{m+1} + a_{m+2} + \dots + a_n| < \varepsilon\) for all \(n > m \ge N\).

Corollary (term test). If \(\sum a_k\) converges then \(a_k \to 0\). Proof: take \(n = m + 1\) in the criterion. \(\blacksquare\) The converse is false, as the harmonic series shows.

Series with non-negative terms

If every \(a_k \ge 0\), the partial sums increase, so by the monotone convergence theorem the series converges if and only if the partial sums are bounded. This simple observation drives several tests.

Theorem (comparison test). Suppose \(0 \le a_k \le b_k\) for all \(k\). If \(\sum b_k\) converges, so does \(\sum a_k\). If \(\sum a_k\) diverges, so does \(\sum b_k\).

Proof. The partial sums of \(\sum a_k\) are bounded by those of \(\sum b_k\), which are bounded by \(\sum b_k\). \(\blacksquare\)

Theorem (\(p\)-series). \(\sum_{k=1}^{\infty} \frac{1}{k^p}\) converges if \(p > 1\) and diverges if \(p \le 1\).

Proof. For \(p \le 1\), \(\frac{1}{k^p} \ge \frac{1}{k}\), so the series diverges by comparison with the harmonic series. For \(p > 1\), group the terms in blocks from \(2^j\) to \(2^{j+1} - 1\). The block has \(2^j\) terms, each at most \(\frac{1}{2^{jp}}\), so its sum is at most \(2^{j(1-p)} = r^j\) with \(r = 2^{1-p} < 1\). The partial sums are therefore bounded by the geometric sum \(\sum_j r^j = \frac{1}{1 - r}\). \(\blacksquare\)

This is a special case of Cauchy's condensation test: for a decreasing non-negative sequence, \(\sum a_k\) converges if and only if \(\sum 2^j a_{2^j}\) does. The value of \(\sum \frac{1}{k^2}\) is \(\frac{\pi^2}{6}\), found by Euler in 1734; this was the famous Basel problem.

The ratio and root tests

Theorem (ratio test). Suppose \(a_k \ne 0\) and \(\big|\frac{a_{k+1}}{a_k}\big| \to \rho\). If \(\rho < 1\) then \(\sum a_k\) converges (absolutely); if \(\rho > 1\) it diverges; if \(\rho = 1\) the test gives no information.

Proof for \(\rho < 1\). Choose \(r\) with \(\rho < r < 1\). For \(k \ge N\), \(\big|\frac{a_{k+1}}{a_k}\big| < r\), so \(|a_{N+j}| \le |a_N| r^j\) by induction. The tail \(\sum_{j \ge 0} |a_{N+j}|\) is dominated by a convergent geometric series, so \(\sum |a_k|\) converges, and hence \(\sum a_k\) converges by the absolute convergence theorem below. For \(\rho > 1\) the terms eventually increase in absolute value, so they do not tend to 0. \(\blacksquare\)

For \(\rho = 1\) both outcomes occur: \(\sum \frac{1}{k}\) diverges and \(\sum \frac{1}{k^2}\) converges, and both have ratio limit 1.

Theorem (root test). Let \(\rho = \limsup |a_k|^{1/k}\). If \(\rho < 1\) the series converges absolutely; if \(\rho > 1\) it diverges. The proof is similar: if \(\rho < r < 1\), then \(|a_k| < r^k\) for large \(k\).

Absolute and conditional convergence

Definition. \(\sum a_k\) converges absolutely if \(\sum |a_k|\) converges. It converges conditionally if it converges but not absolutely.

Theorem. Absolute convergence implies convergence.

Proof. By the triangle inequality \(|a_{m+1} + \dots + a_n| \le |a_{m+1}| + \dots + |a_n|\). If \(\sum |a_k|\) satisfies the Cauchy criterion, so does \(\sum a_k\). \(\blacksquare\)

Theorem (alternating series test). If \((b_k)\) is decreasing and \(b_k \to 0\), then \(\sum_{k=1}^{\infty} (-1)^{k+1} b_k = b_1 - b_2 + b_3 - \dots\) converges, and the error after \(n\) terms is at most \(b_{n+1}\).

Proof. The even partial sums \(s_{2n} = (b_1 - b_2) + \dots + (b_{2n-1} - b_{2n})\) increase, and \(s_{2n} = b_1 - (b_2 - b_3) - \dots - b_{2n} \le b_1\). So \(s_{2n} \to S\) for some \(S\). Then \(s_{2n+1} = s_{2n} + b_{2n+1} \to S\) too. Since both the even and odd subsequences converge to \(S\) and together they contain every term, \(s_n \to S\). The partial sums alternate around \(S\), so \(|S - s_n| \le |s_{n+1} - s_n| = b_{n+1}\). \(\blacksquare\)

The alternating harmonic series \(1 - \frac{1}{2} + \frac{1}{3} - \dots\) therefore converges; its sum is \(\ln 2\), as lesson 11 will show from Taylor's theorem. It converges conditionally, since \(\sum \frac{1}{k}\) diverges.

Rearrangements

Conditionally convergent series are fragile. Riemann's rearrangement theorem, published posthumously in 1868, says that if \(\sum a_k\) converges conditionally then for every real \(T\) there is a rearrangement, a reordering of the terms, whose sum is \(T\).

Sketch of proof. Because the series converges conditionally, the positive terms by themselves sum to \(\infty\) and so do the absolute values of the negative terms, while \(a_k \to 0\). To reach \(T\), add positive terms until the partial sum first exceeds \(T\), then negative terms until it first drops below \(T\), then positive terms again, and so on. Each overshoot is at most the size of the last term used, and those terms tend to 0, so the partial sums converge to \(T\).

By contrast, if \(\sum a_k\) converges absolutely, every rearrangement converges to the same sum. Absolute convergence is the condition under which infinite sums behave like finite ones.

import math

def rearranged(target, n_terms):
    """Riemann's greedy rearrangement of 1 - 1/2 + 1/3 - ... aimed at target."""
    pos = (1 / k for k in range(1, 10**9, 2))      # 1, 1/3, 1/5, ...
    neg = (-1 / k for k in range(2, 10**9, 2))     # -1/2, -1/4, ...
    s, used = 0.0, 0
    while used < n_terms:
        s += next(pos) if s <= target else next(neg)
        used += 1
    return s

print("usual order, 10^6 terms:", round(sum((-1) ** (k + 1) / k for k in range(1, 10**6 + 1)), 6),
      "  ln 2 =", round(math.log(2), 6))
for T in (0.0, 1.5, -1.0, math.pi):
    print(f"rearranged toward {T: .4f}:  after 200000 terms the partial sum is {rearranged(T, 200000): .6f}")

usual order, 10^6 terms: 0.693147   ln 2 = 0.693147
rearranged toward  0.0000:  after 200000 terms the partial sum is -0.000002
rearranged toward  1.5000:  after 200000 terms the partial sum is  1.499992
rearranged toward -1.0000:  after 200000 terms the partial sum is -0.999947
rearranged toward  3.1416:  after 200000 terms the partial sum is  3.141382

Decimal expansions as series

Every decimal expansion is a series. The expression \(0.d_1 d_2 d_3 \dots\) means \(\sum_{k=1}^{\infty} \frac{d_k}{10^k}\) with digits \(0 \le d_k \le 9\). Since \(0 \le \frac{d_k}{10^k} \le \frac{9}{10^k}\) and \(\sum \frac{9}{10^k} = 9 \cdot \frac{1/10}{1 - 1/10} = 1\), the comparison test shows that every decimal expansion converges to a number in \([0, 1]\). This is the precise sense in which an infinite decimal names a real number, and it explains the identity \(0.999\dots = 1\): the left side is the geometric series \(\sum \frac{9}{10^k}\), whose sum is exactly 1.

Conversely, every \(x \in [0, 1)\) has a decimal expansion. Define \(d_1 = \lfloor 10x \rfloor\), then \(d_2 = \lfloor 100x - 10d_1 \rfloor\), and so on. By construction the partial sum \(s_n\) satisfies \(s_n \le x < s_n + 10^{-n}\), so \(|x - s_n| < 10^{-n} \to 0\). The construction depends on the Archimedean property through the existence of the floor function.

Repeating decimals are exactly the rational numbers. If the digits repeat with period \(p\) from some point on, the tail is a geometric series with ratio \(10^{-p}\), whose sum is rational. Conversely, long division of \(a\) by \(b\) can produce only \(b\) different remainders, so the remainders, and hence the digits, eventually repeat. The irrationality of \(\sqrt{2}\) is therefore the statement that its decimal expansion never settles into a cycle.

Why the Cauchy criterion matters in practice

Numerical algorithms rarely know the exact answer they are computing. A program summing a series, or iterating Newton's method, stops when successive approximations agree to within a tolerance. That stopping rule is the Cauchy condition in practice. It is safe only when we also have an estimate of the tail, such as the alternating series bound or a comparison with a geometric series. The harmonic series is a cautionary example: its consecutive partial sums differ by less than \(10^{-6}\) after a million terms, yet the sum is infinite. A rigorous error bound, not merely small consecutive differences, is what justifies stopping.