The inverse function theorem

A differentiable function of one variable with \(f^\prime(a) \ne 0\) and continuous derivative is strictly monotone near \(a\), so it has a local inverse, and \((f^{-1})^\prime(f(a)) = \frac{1}{f^\prime(a)}\). The inverse function theorem says the same is true in several variables, with 'nonzero derivative' replaced by 'invertible derivative'. It is the fundamental theorem about nonlinear equations: if the linear approximation of \(f\) at a point can be inverted, so can \(f\) itself, near that point. The proof combines the contraction mapping theorem of lesson 8 with the mean value inequality of lesson 10.

The statement

Theorem 11.1 (inverse function theorem). Let \(U \subseteq \mathbb{R}^n\) be open, let \(f: U \to \mathbb{R}^n\) be continuously differentiable, and let \(a \in U\) with \(Df(a)\) invertible. Then there are open sets \(W \ni a\) and \(V \ni f(a)\) such that \(f\) maps \(W\) one-to-one onto \(V\), and the inverse \(g = f^{-1}: V \to W\) is continuously differentiable with \[Dg(y) = Df(g(y))^{-1} \quad \text{for } y \in V.\]

A map with these properties, a bijection between open sets which is \(C^1\) with \(C^1\) inverse, is called a \(C^1\) diffeomorphism. The theorem says that \(f\) restricts to a diffeomorphism from a neighborhood of \(a\) onto a neighborhood of \(f(a)\). It is a local statement, and it can only be local, as the examples below show.

The formula for \(Dg\) is forced by the chain rule once we know \(g\) is differentiable: differentiating \(f(g(y)) = y\) gives \(Df(g(y))\,Dg(y) = I\). The real content is the existence of the inverse and its differentiability.

Examples

The complex exponential, written in real coordinates, is \(f(x, y) = (e^x\cos y, \ e^x\sin y)\). Its Jacobian \(\begin{pmatrix} e^x\cos y & -e^x\sin y \\ e^x\sin y & e^x\cos y \end{pmatrix}\) has determinant \(e^{2x} > 0\), so \(Df\) is invertible everywhere, and \(f\) has a local inverse near every point. But \(f(x, y + 2\pi) = f(x, y)\), so \(f\) is not one-to-one on \(\mathbb{R}^2\) and has no global inverse. Locally, the inverse is a branch of the logarithm: \((u, v) \mapsto (\ln\sqrt{u^2 + v^2}, \theta)\) with \(\theta\) a continuous choice of angle.

The cube \(f(x) = x^3\) is one-to-one on \(\mathbb{R}\) with continuous inverse \(x^{1/3}\), but \(f^\prime(0) = 0\) and the inverse is not differentiable at 0. An invertible derivative is sufficient for a differentiable local inverse, and necessary, by the chain rule argument above, but not necessary for a continuous one.

The hypothesis that \(Df\) is continuous cannot simply be dropped. The function \(f(x) = x + 2x^2\sin\frac{1}{x}\), with \(f(0) = 0\), has \(f^\prime(0) = 1\), but \(f^\prime(x) = 1 + 4x\sin\frac{1}{x} - 2\cos\frac{1}{x}\) takes negative values at points arbitrarily close to 0, for example where \(\cos\frac{1}{x} = 1\). So \(f\) is not monotone on any interval around 0 and is not one-to-one on any neighborhood of 0.

Two lemmas

Lemma 11.2. Let \(A\) be an invertible \(n \times n\) matrix and \(B\) a matrix with \(\|B - A\|\,\|A^{-1}\| < 1\). Then \(B\) is invertible.

Proof. It suffices to show \(Bx = 0\) only for \(x = 0\). If \(Bx = 0\), then \(x = A^{-1}Ax = A^{-1}(A - B)x\), so \(|x| \le \|A^{-1}\|\|A - B\||x|\). Since the constant is less than 1, \(|x| = 0\). \(\square\)

Lemma 11.3. The map \(B \mapsto B^{-1}\) is continuous on the set of invertible matrices.

Proof. By Cramer's rule each entry of \(B^{-1}\) is a polynomial in the entries of \(B\) divided by \(\det B\), which is a nonzero polynomial in the entries. Quotients of continuous functions with nonvanishing denominators are continuous. \(\square\)

The proof

Proof of theorem 11.1. Write \(A = Df(a)\), and choose \(\lambda > 0\) with \(2\lambda\|A^{-1}\| = 1\). Since \(Df\) is continuous at \(a\), there is an open ball \(W\) centered at \(a\), contained in \(U\), such that \[\|Df(x) - A\| < \lambda \quad \text{for all } x \in W.\]

Step 1: \(f\) is one-to-one on \(W\). For each \(y \in \mathbb{R}^n\) define \[\phi_y(x) = x + A^{-1}(y - f(x)), \qquad x \in W.\] Then \(f(x) = y\) exactly when \(x\) is a fixed point of \(\phi_y\). Its derivative is \(D\phi_y(x) = I - A^{-1}Df(x) = A^{-1}(A - Df(x))\), so \(\|D\phi_y(x)\| \le \|A^{-1}\|\lambda = \frac{1}{2}\) on \(W\). Since \(W\) is convex, the mean value inequality gives \[|\phi_y(x_1) - \phi_y(x_2)| \le \tfrac{1}{2}|x_1 - x_2| \quad \text{for } x_1, x_2 \in W. \qquad (1)\] So \(\phi_y\) has at most one fixed point in \(W\); that is, at most one \(x \in W\) has \(f(x) = y\).

Step 2: \(V = f(W)\) is open. Let \(y_0 \in V\), say \(y_0 = f(x_0)\) with \(x_0 \in W\), and choose \(r > 0\) such that the closed ball \(\overline{B} = \overline{B}(x_0, r)\) lies in \(W\). We show that every \(y\) with \(|y - y_0| < \lambda r\) is in \(V\). For such \(y\), \[|\phi_y(x_0) - x_0| = |A^{-1}(y - y_0)| \le \|A^{-1}\|\lambda r = \frac{r}{2}.\] For \(x \in \overline{B}\), using (1), \[|\phi_y(x) - x_0| \le |\phi_y(x) - \phi_y(x_0)| + |\phi_y(x_0) - x_0| \le \frac{1}{2}|x - x_0| + \frac{r}{2} \le r.\] So \(\phi_y\) maps \(\overline{B}\) into itself. A closed subset of \(\mathbb{R}^n\) is complete, and \(\phi_y\) is a contraction on it with constant \(\frac{1}{2}\), so by the Banach fixed point theorem it has a fixed point \(x \in \overline{B}\). Then \(f(x) = y\), so \(y \in f(\overline{B}) \subseteq V\). Hence the ball \(B(y_0, \lambda r)\) lies in \(V\), and \(V\) is open.

Step 3: the inverse is differentiable. Let \(g: V \to W\) be the inverse of \(f\) on \(W\), which exists by steps 1 and 2. By lemma 11.2, \(Df(x)\) is invertible for every \(x \in W\), since \(\|Df(x) - A\|\|A^{-1}\| < \lambda\|A^{-1}\| = \frac{1}{2}\). Fix \(y \in V\) and let \(y + k \in V\). Put \(x = g(y)\) and \(h = g(y + k) - x\), so that \(f(x) = y\) and \(f(x + h) = y + k\). From the definition of \(\phi_y\), \[\phi_y(x + h) - \phi_y(x) = h + A^{-1}(f(x) - f(x + h)) = h - A^{-1}k.\] By (1), \(|h - A^{-1}k| \le \frac{1}{2}|h|\), so \(|A^{-1}k| \ge \frac{1}{2}|h|\), and therefore \[|h| \le 2\|A^{-1}\||k| = \frac{|k|}{\lambda}. \qquad (2)\] In particular \(h \to 0\) as \(k \to 0\), so \(g\) is continuous. Let \(T = Df(x)^{-1}\). Then \[g(y + k) - g(y) - Tk = h - Tk = -T\big(f(x + h) - f(x) - Df(x)h\big),\] and by (2), \[\frac{|g(y + k) - g(y) - Tk|}{|k|} \le \frac{\|T\|}{\lambda}\cdot\frac{|f(x + h) - f(x) - Df(x)h|}{|h|}.\] As \(k \to 0\), \(h \to 0\) with \(h \ne 0\) for \(k \ne 0\), and the last fraction tends to 0 by differentiability of \(f\) at \(x\). So \(g\) is differentiable at \(y\) with \(Dg(y) = Df(g(y))^{-1}\).

Step 4: \(g\) is \(C^1\). The map \(y \mapsto Dg(y) = Df(g(y))^{-1}\) is the composition of \(g\), continuous by step 3, of \(Df\), continuous by hypothesis, and of matrix inversion, continuous by lemma 11.3. So it is continuous. \(\square\)

Each ingredient of the course appears in this proof: completeness of closed subsets of \(\mathbb{R}^n\), the contraction mapping theorem, the derivative as a linear map, the operator norm, and the mean value inequality. The contraction \(\phi_y\) is a simplified Newton's method: to solve \(f(x) = y\), it corrects \(x\) by \(A^{-1}(y - f(x))\), using the fixed matrix \(A = Df(a)\) instead of recomputing \(Df(x)\) at each step.

Computing a local inverse

Newton's method, which uses \(Df(x_k)^{-1}\) at each step, converges much faster than the fixed-matrix iteration of the proof when started close enough to a solution. The code inverts the exponential map \(f(x, y) = (e^x\cos y, e^x\sin y)\) near \(a = (0, 1)\) both ways, and compares the derivative of the inverse with \(Df(g(y))^{-1}\) and with a finite difference.

import math

def f(p):
    x, y = p
    return [math.exp(x) * math.cos(y), math.exp(x) * math.sin(y)]

def Df(p):
    x, y = p
    e = math.exp(x)
    return [[e * math.cos(y), -e * math.sin(y)], [e * math.sin(y), e * math.cos(y)]]

def inv2(M):
    (a, b), (c, d) = M
    det = a * d - b * c
    return [[d / det, -b / det], [-c / det, a / det]]

def mat_vec(M, v):
    return [M[0][0] * v[0] + M[0][1] * v[1], M[1][0] * v[0] + M[1][1] * v[1]]

a = [0.0, 1.0]
Ainv = inv2(Df(a))
target = [0.3, 1.0]                       # solve f(x) = target, starting from a

for name, use_newton in [("fixed matrix (proof)", False), ("Newton", True)]:
    x = list(a)
    print(name)
    for k in range(1, 8):
        r = [t - s for t, s in zip(target, f(x))]
        M = inv2(Df(x)) if use_newton else Ainv
        step = mat_vec(M, r)
        x = [x[0] + step[0], x[1] + step[1]]
        res = math.hypot(*[t - s for t, s in zip(target, f(x))])
        print(f"  step {k}: x = ({x[0]:.12f}, {x[1]:.12f})   |f(x) - y| = {res:.2e}")

exact = [math.log(math.hypot(*target)), math.atan2(target[1], target[0])]
print("exact inverse: (", round(exact[0], 12), ",", round(exact[1], 12), ")")

# derivative of the inverse: Df(g(y))^(-1) versus central differences of g = (log r, angle)
g = lambda u: [math.log(math.hypot(*u)), math.atan2(u[1], u[0])]
Dg_formula = inv2(Df(exact))
t = 1e-6
Dg_numeric = [[(g([target[0] + t, target[1]])[i] - g([target[0] - t, target[1]])[i]) / (2 * t),
               (g([target[0], target[1] + t])[i] - g([target[0], target[1] - t])[i]) / (2 * t)] for i in range(2)]
print("Dg from formula:", [[round(v, 8) for v in row] for row in Dg_formula])
print("Dg numerically: ", [[round(v, 8) for v in row] for row in Dg_numeric])

fixed matrix (proof)
  step 1: x = (0.003561676568, 1.287861010426)   |f(x) - y| = 4.14e-02
  step 2: x = (0.044848806919, 1.290807152393)   |f(x) - y| = 1.21e-02
  step 3: x = (0.046453950525, 1.278790560094)   |f(x) - y| = 3.57e-03
  step 4: x = (0.042912765578, 1.278373074659)   |f(x) - y| = 1.03e-03
  step 5: x = (0.042811771862, 1.279393620858)   |f(x) - y| = 2.95e-04
  step 6: x = (0.043105363594, 1.279419103533)   |f(x) - y| = 8.48e-05
  step 7: x = (0.043111698167, 1.279334494303)   |f(x) - y| = 2.44e-05
Newton
  step 1: x = (0.003561676568, 1.287861010426)   |f(x) - y| = 4.14e-02
  step 2: x = (0.043842670693, 1.278996064129)   |f(x) - y| = 8.65e-04
  step 3: x = (0.043089073233, 1.279339273494)   |f(x) - y| = 3.58e-07
  step 4: x = (0.043088848121, 1.279339532317)   |f(x) - y| = 6.13e-14
  step 5: x = (0.043088848121, 1.279339532317)   |f(x) - y| = 0.00e+00
  step 6: x = (0.043088848121, 1.279339532317)   |f(x) - y| = 0.00e+00
  step 7: x = (0.043088848121, 1.279339532317)   |f(x) - y| = 0.00e+00
exact inverse: ( 0.043088848121 , 1.279339532317 )
Dg from formula: [[0.27522936, 0.91743119], [-0.91743119, 0.27522936]]
Dg numerically:  [[0.27522936, 0.91743119], [-0.91743119, 0.27522936]]

The fixed-matrix iteration of the proof converges at a steady geometric rate, the residual shrinking by a factor of about 3.5 per step, as a contraction should. Newton's method roughly doubles the number of correct digits each step and reaches the limits of floating point within a few steps. Both find the same point, \((\ln|y|, \arg y)\), the local logarithm, and the derivative of the inverse computed from \(Df(g(y))^{-1}\) agrees with a direct finite-difference computation to about eight decimal places.

Consequences and history

Corollary 11.4 (open mapping). If \(f: U \to \mathbb{R}^n\) is \(C^1\) and \(Df(x)\) is invertible for every \(x \in U\), then \(f(O)\) is open for every open \(O \subseteq U\). Proof. Each point \(f(x)\) with \(x \in O\) has, by the theorem applied to \(f\) restricted to \(O\), an open neighborhood \(V\) contained in \(f(O)\). \(\square\)

A map with invertible derivative everywhere is a local diffeomorphism. It need not be globally one-to-one, as the exponential map shows. Hadamard proved in 1906 that a \(C^1\) local diffeomorphism \(f: \mathbb{R}^n \to \mathbb{R}^n\) is a global diffeomorphism if it is proper, meaning \(|f(x)| \to \infty\) as \(|x| \to \infty\). In one variable no such condition is needed, since a continuous derivative that never vanishes has constant sign.

The inverse function theorem grew out of the implicit function theorem of lesson 12, whose history is older. In the 1870s Ulisse Dini gave, in lectures at Pisa, the first proof of the implicit function theorem for real functions of several variables, and the inverse function theorem followed as a corollary. Édouard Goursat gave in 1903 a proof by successive approximations, an early instance of the idea later formalized in Banach's fixed point theorem, and in 1927 Theophil Hildebrandt and Lawrence Graves extended both theorems to maps between Banach spaces, where the proof above works almost word for word. Today the inverse function theorem is the foundation of the theory of manifolds, studied in MATH-430.