Taylor's theorem and L'Hôpital's rule

The mean value theorem says that \(f(b) = f(a) + f'(c)(b - a)\) for some \(c\) between \(a\) and \(b\): a linear approximation with an error controlled by the derivative. Taylor's theorem extends this to approximations by polynomials of any degree, with an error controlled by a higher derivative. It turns statements such as \(e \approx 2.71828\) or \(\ln 2 \approx 0.693\) into theorems with explicit error bounds. The same circle of ideas, through Cauchy's generalized mean value theorem, proves L'Hôpital's rule for indeterminate limits. This lesson proves both, and uses them to show that \(e\) is irrational and that the alternating harmonic series sums to \(\ln 2\).

L'Hôpital's rule

Theorem (L'Hôpital, 0/0 form). Let \(f\) and \(g\) be differentiable on an open interval containing \(c\), except possibly at \(c\), with \(g'(x) \ne 0\) for \(x \ne c\). Suppose \(\lim_{x \to c} f(x) = \lim_{x \to c} g(x) = 0\) and \(\lim_{x \to c} \frac{f'(x)}{g'(x)} = L\). Then \(\lim_{x \to c} \frac{f(x)}{g(x)} = L\).

Proof. Redefine \(f(c) = g(c) = 0\); this makes \(f\) and \(g\) continuous at \(c\) without changing the limits. Let \(x > c\) in the interval. On \([c, x]\), Cauchy's mean value theorem (lesson 10) gives \(t\) with \(c < t < x\) and \((f(x) - f(c))g'(t) = (g(x) - g(c))f'(t)\), that is \(f(x)g'(t) = g(x)f'(t)\). Here \(g(x) \ne 0\), since otherwise Rolle would give a zero of \(g'\) in \((c, x)\). So \(\frac{f(x)}{g(x)} = \frac{f'(t)}{g'(t)}\). As \(x \to c^+\), the point \(t\), squeezed between \(c\) and \(x\), tends to \(c\), so the right side tends to \(L\). The case \(x < c\) is identical. \(\blacksquare\)

The rule is named after the Marquis de L'Hôpital, whose 1696 book Analyze des infiniment petits was the first calculus textbook. The result itself was discovered by Johann Bernoulli, whom L'Hôpital paid for his mathematical discoveries. Versions also hold for \(x \to \infty\) and for the \(\frac{\infty}{\infty}\) form, where \(|g(x)| \to \infty\); the proof of the latter is more delicate and is given in MATH-341.

Taylor polynomials

If \(f\) has derivatives up to order \(n\) at \(a\), its Taylor polynomial of degree \(n\) at \(a\) is

\[P_n(x) = \sum_{k=0}^{n} \frac{f^{(k)}(a)}{k!}(x - a)^k = f(a) + f'(a)(x - a) + \frac{f''(a)}{2}(x - a)^2 + \dots + \frac{f^{(n)}(a)}{n!}(x - a)^n.\]

It is the unique polynomial of degree at most \(n\) whose value and first \(n\) derivatives at \(a\) agree with those of \(f\), as differentiating term by term shows. For \(f(x) = e^x\) at \(a = 0\), every derivative is 1 at 0, so \(P_n(x) = \sum_{k=0}^{n} \frac{x^k}{k!}\). For \(\ln(1 + x)\) at 0, \(f^{(k)}(0) = (-1)^{k+1}(k - 1)!\), so \(P_n(x) = x - \frac{x^2}{2} + \frac{x^3}{3} - \dots + (-1)^{n+1}\frac{x^n}{n}\).

The question is how well \(P_n\) approximates \(f\). The remainder is \(R_n(x) = f(x) - P_n(x)\).

Taylor's theorem

Theorem (Taylor, Lagrange remainder). Let \(f\) have \(n + 1\) derivatives on an open interval \(I\) containing \(a\). For every \(x \in I\) there is \(c\) between \(a\) and \(x\) with

\[f(x) = P_n(x) + \frac{f^{(n+1)}(c)}{(n+1)!}(x - a)^{n+1}.\]

Proof. Fix \(x \ne a\) and choose the number \(M\) so that \(f(x) = P_n(x) + M(x - a)^{n+1}\). We must show \(M = \frac{f^{(n+1)}(c)}{(n+1)!}\) for some \(c\). Define \(g(t) = f(t) - P_n(t) - M(t - a)^{n+1}\). Then \(g(x) = 0\) by the choice of \(M\), and \(g^{(k)}(a) = 0\) for \(k = 0, 1, \dots, n\), because \(P_n\) matches the first \(n\) derivatives of \(f\) at \(a\) and \((t - a)^{n+1}\) has all its first \(n\) derivatives zero at \(a\).

Apply Rolle repeatedly. Since \(g(a) = g(x) = 0\), there is \(x_1\) strictly between \(a\) and \(x\) with \(g'(x_1) = 0\). Since \(g'(a) = g'(x_1) = 0\), there is \(x_2\) between \(a\) and \(x_1\) with \(g''(x_2) = 0\). Continuing, we get \(x_{n+1}\) between \(a\) and \(x\) with \(g^{(n+1)}(x_{n+1}) = 0\). But \(P_n^{(n+1)} = 0\), since \(P_n\) has degree at most \(n\), and the \((n + 1)\)-th derivative of \(M(t - a)^{n+1}\) is \((n+1)!\,M\). So \(0 = f^{(n+1)}(c) - (n+1)!\,M\) with \(c = x_{n+1}\). \(\blacksquare\)

For \(n = 0\) this is the mean value theorem. Lagrange stated the remainder in this form in 1797. Brook Taylor had published the series in 1715, without any discussion of convergence; Colin Maclaurin popularized the case \(a = 0\).

Error bounds in practice

Theorem. \(e\) is irrational.

Proof (after Fourier). Suppose \(e = \frac{p}{q}\) with positive integers \(p, q\). Taylor's theorem with \(n = q\) gives \(e = \sum_{k=0}^{q} \frac{1}{k!} + \frac{e^c}{(q+1)!}\) with \(0 < c < 1\). Multiply by \(q!\): the number \(q!\,e - \sum_{k=0}^{q} \frac{q!}{k!}\) is an integer, since \(q!\,e = (q-1)!\,p\) and each \(\frac{q!}{k!}\) is an integer. But it equals \(\frac{e^c}{q + 1}\), which lies strictly between 0 and \(\frac{3}{q+1}\). If \(q \ge 2\) then \(\frac{3}{q+1} \le 1\), and no integer lies strictly between 0 and 1. If \(q = 1\), \(e\) would be an integer, but \(2 < e < 3\). Either way we have a contradiction. \(\blacksquare\)

When Taylor series fail

If \(f\) has derivatives of all orders and \(R_n(x) \to 0\), then \(f(x) = \sum_{k=0}^{\infty} \frac{f^{(k)}(a)}{k!}(x - a)^k\). For \(e^x\), \(\sin x\) and \(\cos x\) this holds for every \(x\), since \(\frac{|x|^{n+1}}{(n+1)!} \to 0\). But the remainder need not tend to zero, even for very smooth functions.

Cauchy's example. Let \(f(x) = e^{-1/x^2}\) for \(x \ne 0\) and \(f(0) = 0\). One shows by induction that for \(x \ne 0\), \(f^{(k)}(x) = p_k(\frac{1}{x}) e^{-1/x^2}\) for a polynomial \(p_k\), and that \(f^{(k)}(0) = 0\) for every \(k\), using the fact that \(t^m e^{-t^2} \to 0\) as \(t \to \infty\). So every Taylor polynomial of \(f\) at 0 is identically zero, while \(f(x) > 0\) for \(x \ne 0\). The Taylor series converges everywhere, to the wrong function. Functions equal to their Taylor series near each point are called analytic; this \(f\) is smooth but not analytic at 0. Such functions are indispensable in analysis because they can switch on smoothly, which analytic functions cannot.

The second derivative test

Taylor's theorem with \(n = 1\) gives the second derivative test. If \(f'(a) = 0\) and \(f''\) is continuous with \(f''(a) > 0\), then for \(x\) near \(a\), \(f(x) = f(a) + \frac{f''(c)}{2}(x - a)^2\) with \(c\) between \(a\) and \(x\), and \(f''(c) > 0\) by continuity. So \(f(x) > f(a)\) for \(x \ne a\) near \(a\): a strict local minimum. More generally, if the first non-zero derivative at a critical point has even order, the point is a local extremum; if it has odd order, it is not, as \(x^3\) at 0 shows.

from fractions import Fraction as F
import math

# e from its Taylor polynomials, with the Lagrange bound 3/(n+1)!
s = F(0)
for n in range(0, 13):
    s += F(1, math.factorial(n))
    if n in (2, 4, 6, 8, 10, 12):
        bound = F(3, math.factorial(n + 1))
        print(f"n = {n:2}: P_n(1) = {float(s):.12f}   true error = {math.e - float(s):.2e}   bound = {float(bound):.2e}")

# ln 2 from the alternating harmonic series, with the bound 1/(n+1)
print()
t = F(0)
for n in range(1, 10001):
    t += F((-1) ** (n + 1), n)
    if n in (10, 100, 1000, 10000):
        print(f"n = {n:5}: partial sum = {float(t):.8f}   true error = {abs(math.log(2) - float(t)):.2e}   bound = {1 / (n + 1):.2e}")

n =  2: P_n(1) = 2.500000000000   true error = 2.18e-01   bound = 5.00e-01
n =  4: P_n(1) = 2.708333333333   true error = 9.95e-03   bound = 2.50e-02
n =  6: P_n(1) = 2.718055555556   true error = 2.26e-04   bound = 5.95e-04
n =  8: P_n(1) = 2.718278769841   true error = 3.06e-06   bound = 8.27e-06
n = 10: P_n(1) = 2.718281801146   true error = 2.73e-08   bound = 7.52e-08
n = 12: P_n(1) = 2.718281828286   true error = 1.73e-10   bound = 4.82e-10

n =    10: partial sum = 0.64563492   true error = 4.75e-02   bound = 9.09e-02
n =   100: partial sum = 0.68817218   true error = 4.98e-03   bound = 9.90e-03
n =  1000: partial sum = 0.69264743   true error = 5.00e-04   bound = 9.99e-04
n = 10000: partial sum = 0.69309718   true error = 5.00e-05   bound = 1.00e-04

The two series show the two extremes. The error for \(e\) falls factorially and soon reaches the limits of floating-point arithmetic, while the error for \(\ln 2\) falls only like \(\frac{1}{n}\). Faster series for \(\ln 2\) come from applying Taylor's theorem at a point closer to the center, for instance \(\ln 2 = -\ln(1 - \frac{1}{2}) = \sum_{k=1}^{\infty} \frac{1}{k 2^k}\), whose terms fall geometrically.

Taylor expansions instead of L'Hôpital

Repeated use of L'Hôpital's rule can be replaced by Taylor's theorem, which often shows more clearly why a limit has its value. Write \(f(x) = P_n(x) + R_n(x)\) where \(|R_n(x)| \le C|x - a|^{n+1}\) near \(a\), whenever \(f^{(n+1)}\) is bounded there. For example \(\sin x = x - \frac{x^3}{6} + R(x)\) with \(|R(x)| \le \frac{|x|^5}{120}\), so

\[\frac{x - \sin x}{x^3} = \frac{1}{6} - \frac{R(x)}{x^3}, \qquad \Big|\frac{R(x)}{x^3}\Big| \le \frac{x^2}{120} \to 0.\]

The limit is \(\frac{1}{6}\), and the expansion also says how fast the quotient approaches it. Three applications of L'Hôpital's rule reach the same answer with less insight.

The \(\frac{\infty}{\infty}\) form handles limits at infinity such as \(\lim_{x \to \infty} \frac{x^m}{e^x} = 0\) for every \(m\): after \(m\) applications the quotient is \(\frac{m!}{e^x} \to 0\). Taylor's theorem gives a direct proof as well. For \(x > 0\) every term of the Taylor polynomial of \(e^x\) is positive, and the remainder is positive, so \(e^x > \frac{x^{m+1}}{(m+1)!}\). Hence \(0 < \frac{x^m}{e^x} < \frac{(m+1)!}{x} \to 0\). This is the continuous version of the fact, proved with the ratio test in lesson 6, that exponentials grow faster than every polynomial.