The derivative and the mean value theorem

The derivative is a limit, so everything in lessons 7 to 9 now applies to it. This lesson defines the derivative precisely, proves that differentiable functions are continuous, and proves the product, quotient and chain rules. It then proves the mean value theorem, the result that connects the derivative at single points to the behaviour of a function over an interval. Most of the facts about derivatives used in calculus, such as that a function with positive derivative is increasing, are consequences of this theorem. The lesson ends with derivatives that behave strangely: a derivative that exists everywhere but is discontinuous, and Weierstrass's continuous function that has no derivative anywhere.

The definition

Definition. Let \(I\) be an interval, \(f: I \to \mathbb{R}\) and \(c \in I\). Then \(f\) is differentiable at \(c\) if the limit

\[f'(c) = \lim_{x \to c} \frac{f(x) - f(c)}{x - c}\]

exists. The difference quotient is not defined at \(x = c\), which is why the limit must be taken in the sense of lesson 7, ignoring the point itself. At an endpoint of \(I\) the limit is one-sided.

Equivalently, with \(x = c + h\), \(f'(c) = \lim_{h \to 0} \frac{f(c + h) - f(c)}{h}\). Geometrically \(f'(c)\) is the slope of the tangent line, the limit of slopes of secant lines through \((c, f(c))\).

Theorem. If \(f\) is differentiable at \(c\), then \(f\) is continuous at \(c\).

Proof. For \(x \ne c\), \(f(x) - f(c) = \frac{f(x) - f(c)}{x - c} \cdot (x - c)\). As \(x \to c\) the first factor tends to \(f'(c)\) and the second to 0, so \(f(x) \to f(c)\). \(\blacksquare\)

The converse fails, as \(|x|\) shows. It fails very badly, as we see at the end of the lesson.

The rules of differentiation

Theorem. If \(f\) and \(g\) are differentiable at \(c\), then so are \(f + g\), \(fg\) and, if \(g(c) \ne 0\), \(\frac{f}{g}\), with

\[(fg)'(c) = f'(c)g(c) + f(c)g'(c), \qquad \Big(\frac{f}{g}\Big)'(c) = \frac{f'(c)g(c) - f(c)g'(c)}{g(c)^2}.\]

Proof of the product rule. Add and subtract \(f(c)g(x)\):

\[\frac{f(x)g(x) - f(c)g(c)}{x - c} = \frac{f(x) - f(c)}{x - c}\, g(x) + f(c)\, \frac{g(x) - g(c)}{x - c}.\]

As \(x \to c\), \(g(x) \to g(c)\) because \(g\) is continuous at \(c\), and the two quotients tend to \(f'(c)\) and \(g'(c)\). \(\blacksquare\) The quotient rule is proved the same way, using \(g(x) \ne 0\) for \(x\) near \(c\).

Theorem (chain rule). If \(g\) is differentiable at \(c\) and \(f\) is differentiable at \(g(c)\), then \(f \circ g\) is differentiable at \(c\) with \((f \circ g)'(c) = f'(g(c))\, g'(c)\).

The tempting proof writes \(\frac{f(g(x)) - f(g(c))}{x - c} = \frac{f(g(x)) - f(g(c))}{g(x) - g(c)} \cdot \frac{g(x) - g(c)}{x - c}\), but \(g(x) - g(c)\) may be zero for \(x\) arbitrarily close to \(c\), so the first fraction can be undefined. The fix, due in this form to Carathéodory, is to package the difference quotient of \(f\) as a continuous function.

Proof. Let \(d = g(c)\) and define \(\varphi(y) = \frac{f(y) - f(d)}{y - d}\) for \(y \ne d\) and \(\varphi(d) = f'(d)\). Then \(\varphi\) is continuous at \(d\), and \(f(y) - f(d) = \varphi(y)(y - d)\) for every \(y\), including \(y = d\). Substituting \(y = g(x)\) and dividing by \(x - c\):

\[\frac{f(g(x)) - f(g(c))}{x - c} = \varphi(g(x)) \cdot \frac{g(x) - g(c)}{x - c}.\]

As \(x \to c\), \(g(x) \to d\) by continuity of \(g\), so \(\varphi(g(x)) \to \varphi(d) = f'(d)\) by continuity of \(\varphi\) at \(d\), and the second factor tends to \(g'(c)\). \(\blacksquare\)

Interior extrema and Rolle's theorem

Theorem (interior extremum). If \(f\) is differentiable at an interior point \(c\) of \(I\) and has a local maximum or minimum at \(c\), then \(f'(c) = 0\).

Proof. Suppose \(f(x) \le f(c)\) for \(x\) near \(c\). For \(x > c\) near \(c\), \(\frac{f(x) - f(c)}{x - c} \le 0\), so the right-hand limit gives \(f'(c) \le 0\). For \(x < c\) the quotient is \(\ge 0\), giving \(f'(c) \ge 0\). Hence \(f'(c) = 0\). \(\blacksquare\) This result is due to Fermat in spirit, from his method of maxima of the 1630s.

Theorem (Rolle). If \(f\) is continuous on \([a, b]\), differentiable on \((a, b)\), and \(f(a) = f(b)\), then \(f'(c) = 0\) for some \(c \in (a, b)\).

Proof. By the extreme value theorem, \(f\) attains a maximum and a minimum on \([a, b]\). If both occur at the endpoints, then since \(f(a) = f(b)\), \(f\) is constant and \(f' = 0\) everywhere. Otherwise one of them occurs at an interior point \(c\), and \(f'(c) = 0\) by the interior extremum theorem. \(\blacksquare\)

Michel Rolle stated the result for polynomials in 1691. Its proof here rests on the extreme value theorem and hence on compactness and completeness: over \(\mathbb{Q}\), the function \(x^3 - x\) vanishes at 0 and at 1 but has no rational critical point at all, since \(3x^2 = 1\) has no rational solution.

The mean value theorem

Theorem (mean value theorem). If \(f\) is continuous on \([a, b]\) and differentiable on \((a, b)\), there is \(c \in (a, b)\) with

\[f'(c) = \frac{f(b) - f(a)}{b - a}.\]

Proof. Subtract the secant line: let \(h(x) = f(x) - f(a) - \frac{f(b) - f(a)}{b - a}(x - a)\). Then \(h\) is continuous on \([a, b]\), differentiable on \((a, b)\), and \(h(a) = h(b) = 0\). By Rolle there is \(c\) with \(0 = h'(c) = f'(c) - \frac{f(b) - f(a)}{b - a}\). \(\blacksquare\)

The theorem is due to Lagrange in 1797 and was given a careful proof by Cauchy in 1823, who also proved the generalized form: if \(f\) and \(g\) are continuous on \([a, b]\) and differentiable on \((a, b)\), there is \(c\) with \((f(b) - f(a))g'(c) = (g(b) - g(a))f'(c)\). The proof applies Rolle to \(h(x) = (f(b) - f(a))g(x) - (g(b) - g(a))f(x)\). Lesson 11 uses this form to prove L'Hôpital's rule.

Consequences

Corollary 1. If \(f'(x) = 0\) for every \(x \in (a, b)\), then \(f\) is constant on \((a, b)\). Proof: for \(x < y\) in \((a, b)\), \(f(y) - f(x) = f'(c)(y - x) = 0\). \(\blacksquare\)

Corollary 2. If \(f' = g'\) on an interval, then \(f - g\) is constant. This is why antiderivatives are unique up to a constant, the fact behind the \(+ C\) of calculus. On a set that is not an interval it fails: \(\operatorname{sgn}(x)\) has derivative 0 on \(\mathbb{R} \setminus \{0\}\) but is not constant.

Corollary 3. If \(f' > 0\) on an interval, \(f\) is strictly increasing there; if \(f' \ge 0\), it is increasing. Proof: \(f(y) - f(x) = f'(c)(y - x)\) has the sign of \(f'(c)\). \(\blacksquare\) The converse of the strict version fails: \(x^3\) is strictly increasing with \(f'(0) = 0\).

Corollary 4. If \(|f'(x)| \le K\) on an interval, then \(|f(x) - f(y)| \le K|x - y|\), so \(f\) is Lipschitz and hence uniformly continuous. For example \(|\sin x - \sin y| \le |x - y|\).

Strange derivatives

A derivative need not be continuous. Let \(f(x) = x^2 \sin\frac{1}{x}\) for \(x \ne 0\) and \(f(0) = 0\). At 0, \(\big|\frac{f(h) - f(0)}{h}\big| = |h \sin\frac{1}{h}| \le |h| \to 0\), so \(f'(0) = 0\). For \(x \ne 0\), the rules give \(f'(x) = 2x\sin\frac{1}{x} - \cos\frac{1}{x}\). As \(x \to 0\) the first term tends to 0 while \(\cos\frac{1}{x}\) oscillates between \(-1\) and 1, so \(f'\) has no limit at 0. The function is differentiable everywhere, but its derivative is discontinuous at 0.

Even so, derivatives cannot jump. Darboux's theorem (1875) says that if \(f\) is differentiable on \([a, b]\) and \(y\) lies strictly between \(f'(a)\) and \(f'(b)\), then \(f'(c) = y\) for some \(c \in (a, b)\). Proof sketch: assume \(f'(a) < y < f'(b)\) and let \(g(x) = f(x) - yx\), so \(g'(a) < 0 < g'(b)\). Then \(g\) takes values below \(g(a)\) just to the right of \(a\) and below \(g(b)\) just to the left of \(b\), so its minimum on \([a, b]\) is at an interior point \(c\), where \(g'(c) = 0\). Consequently a function with a jump discontinuity, such as \(\operatorname{sgn}\), is not the derivative of anything.

A continuous function need not be differentiable anywhere. In 1872 Weierstrass presented to the Berlin Academy the function

\[W(x) = \sum_{n=0}^{\infty} a^n \cos(b^n \pi x), \qquad 0 < a < 1,\ b \text{ an odd integer},\ ab > 1 + \tfrac{3\pi}{2},\]

and proved it is continuous everywhere and differentiable nowhere. The series converges uniformly by comparison with \(\sum a^n\), which gives continuity (MATH-341 proves that uniform limits of continuous functions are continuous). The partial sums are smooth, but each new term adds oscillations that are \(b\) times faster and only \(a\) times smaller, so the slopes they contribute grow like \((ab)^n\). Bolzano had constructed a similar function in the 1830s, but his work was not published until 1922. Weierstrass's example ended the belief, common among earlier analysts, that a continuous function must be differentiable except at isolated points.

import math

a, b = 0.5, 13            # ab = 6.5 > 1 + 3*pi/2, about 5.71
def W(x, terms):
    return sum(a**n * math.cos(b**n * math.pi * x) for n in range(terms))

def slope(f, x, h):
    return (f(x + h) - f(x)) / h

x0 = 0.3
print("difference quotients of the partial sums at x = 0.3 (terms = number of terms kept):")
for h in (1e-2, 1e-3, 1e-4, 1e-5):
    row = "  ".join(f"{slope(lambda t: W(t, k), x0, h):11.2f}" for k in (1, 2, 3, 4, 6))
    print(f"h = {h:.0e}:", row)
print("columns: terms = 1, 2, 3, 4, 6")

difference quotients of the partial sums at x = 0.3 (terms = number of terms kept):
h = 1e-02:       -2.57        -0.34        22.82        22.51        19.17
h = 1e-03:       -2.54         3.37       -78.81       -34.35      -183.12
h = 1e-04:       -2.54         3.73      -101.53       416.51     -1110.58
h = 1e-05:       -2.54         3.76      -103.41       191.29     -7029.60
columns: terms = 1, 2, 3, 4, 6

For a single smooth partial sum the difference quotients settle down once \(h\) is small. For more terms they settle only at smaller and smaller \(h\), and at much larger values. In the limit there is no scale at which the function looks like a straight line, which is what nowhere differentiable means.

Derivatives of inverse functions

Theorem. Let \(f\) be continuous and strictly monotone on an interval, with inverse \(g\). If \(f\) is differentiable at \(c\) with \(f'(c) \ne 0\), then \(g\) is differentiable at \(d = f(c)\) and \(g'(d) = \frac{1}{f'(c)}\).

Proof. By lesson 8, \(g\) is continuous. For \(y \ne d\) in the domain of \(g\), put \(x = g(y)\); then \(x \ne c\) because \(g\) is injective, and \(\frac{g(y) - g(d)}{y - d} = \frac{x - c}{f(x) - f(c)}\). As \(y \to d\), continuity of \(g\) gives \(x \to c\), so the quotient tends to \(\frac{1}{f'(c)}\). \(\blacksquare\)

For example, \(\sqrt[n]{y}\) is the inverse of \(x^n\) on \((0, \infty)\), so its derivative at \(y = x^n\) is \(\frac{1}{nx^{n-1}} = \frac{1}{n} y^{\frac{1}{n} - 1}\), the expected power rule. The condition \(f'(c) \ne 0\) is necessary: \(x^3\) has \(f'(0) = 0\), and its inverse \(\sqrt[3]{y}\) has a vertical tangent at 0, where it is not differentiable.