Jordan canonical form

Not every operator is diagonalizable, but every complex operator is almost diagonalizable. The Jordan canonical form says that in a suitable basis the matrix is block diagonal, each block having one eigenvalue on its diagonal and ones just above it. The form is unique up to the order of the blocks, so it answers completely the question lesson 4 raised: two complex matrices are similar exactly when they have the same Jordan form. This lesson proves the theorem in two steps, first splitting the space into generalized eigenspaces and then analysing nilpotent operators, and shows how to read the block sizes from ranks without computing any basis at all. We finish with powers and exponentials of Jordan blocks and a frank discussion of why numerical software avoids the Jordan form.

Jordan blocks

A Jordan block of size \(k\) with eigenvalue \(\lambda\) is the \(k \times k\) matrix

where \(N_k\) has ones on the superdiagonal and zeros elsewhere. The matrix \(N_k\) is the shift \(\mathbf{e}_k \mapsto \mathbf{e}_{k-1} \mapsto \dots \mapsto \mathbf{e}_1 \mapsto \mathbf{0}\), so \(N_k^k = 0\) but \(N_k^{k-1} \ne 0\). A Jordan block has a single eigenvalue \(\lambda\) of algebraic multiplicity \(k\), a one-dimensional eigenspace spanned by \(\mathbf{e}_1\), and minimal polynomial \((x - \lambda)^k\). It is the most non-diagonalizable an operator on a \(k\)-dimensional space can be.

Theorem (Jordan canonical form). Let \(T\) be an operator on a finite-dimensional complex vector space. There is a basis in which the matrix of \(T\) is block diagonal with Jordan blocks on the diagonal:

The eigenvalues \(\lambda_i\) need not be distinct. The list of blocks, with their sizes and eigenvalues, is uniquely determined by \(T\) up to reordering.

Step one: generalized eigenspaces

A vector \(\mathbf{v}\) is a generalized eigenvector for \(\lambda\) if \((T - \lambda I)^r\mathbf{v} = \mathbf{0}\) for some \(r \ge 1\). The generalized eigenspace is \(G_\lambda = \ker(T - \lambda I)^n\), where \(n = \dim V\); taking the exponent \(n\) is enough because the kernels \(\ker(T - \lambda I)^r\) increase with \(r\) and, once two consecutive ones are equal, stay equal forever.

Theorem. If \(\lambda_1, \dots, \lambda_k\) are the distinct eigenvalues of \(T\) over \(\mathbb{C}\), then \(V = G_{\lambda_1} \oplus \dots \oplus G_{\lambda_k}\), each \(G_{\lambda_i}\) is invariant, \(\dim G_{\lambda_i}\) equals the algebraic multiplicity of \(\lambda_i\), and \(T - \lambda_iI\) is nilpotent on \(G_{\lambda_i}\).

Proof sketch. Write \(p_T(x) = \prod_i(x - \lambda_i)^{a_i}\) and let \(q_i = p_T/(x - \lambda_i)^{a_i}\). These polynomials have no common factor, so by the Euclidean algorithm there are polynomials \(s_i\) with \(s_1q_1 + \dots + s_kq_k = 1\). Applying this identity to \(T\) gives \(\mathbf{v} = \sum_i s_i(T)q_i(T)\mathbf{v}\), and Cayley–Hamilton shows that the \(i\)th piece is killed by \((T - \lambda_iI)^{a_i}\), so lies in \(G_{\lambda_i}\). Thus the generalized eigenspaces span \(V\). The sum is direct because \(T - \lambda_jI\) is invertible on \(G_{\lambda_i}\) for \(j \ne i\), which lets the argument of lesson 5 go through. Each \(G_{\lambda_i}\) is the kernel of a polynomial in \(T\), so it is invariant. On \(G_{\lambda_i}\) the only eigenvalue is \(\lambda_i\), so the dimension count follows from the factorization of the characteristic polynomial along the direct sum.

After this step the matrix of \(T\) is block diagonal with one block per distinct eigenvalue, and each block is \(\lambda_iI\) plus a nilpotent matrix. It remains to put a nilpotent operator in standard form.

Step two: nilpotent operators

Let \(N\) be nilpotent on a space \(W\). For a vector \(\mathbf{w}\) with \(N^{k}\mathbf{w} = \mathbf{0}\) but \(N^{k-1}\mathbf{w} \ne \mathbf{0}\), the list \(N^{k-1}\mathbf{w}, \dots, N\mathbf{w}, \mathbf{w}\) is called a Jordan chain of length \(k\). It is linearly independent (apply \(N^{k-1}\) to a dependence relation to see that the coefficient of \(\mathbf{w}\) is zero, then repeat), and in this ordered basis of its span, \(N\) acts exactly as \(N_k\): each vector goes to the one before it and the first goes to zero.

Theorem. For any nilpotent \(N\) on \(W\), there are vectors \(\mathbf{w}_1, \dots, \mathbf{w}_s\) whose Jordan chains together form a basis of \(W\). In that basis \(N\) is block diagonal with blocks \(N_{k_1}, \dots, N_{k_s}\).

Proof by induction on \(\dim W\). The image \(NW\) is a proper invariant subspace (proper because \(N\) is not injective), so by induction it has a basis of chains, generated by \(\mathbf{u}_1, \dots, \mathbf{u}_t\). Each \(\mathbf{u}_i\) lies in the image, so choose \(\mathbf{w}_i\) with \(N\mathbf{w}_i = \mathbf{u}_i\); each chain gets one vector longer. The last vectors of the chains in \(NW\) form a basis of \(NW \cap \ker N\); extend it to a basis of \(\ker N\) by vectors \(\mathbf{z}_1, \dots, \mathbf{z}_r\), each of which is a chain of length 1. Counting shows the lengthened chains plus the \(\mathbf{z}_j\) have \(\dim NW + t + r = \dim NW + \dim\ker N = \dim W\) vectors, by rank–nullity, and an independence argument like the one for a single chain shows they are independent. So they form a basis.

Uniqueness: block sizes from ranks

How many Jordan blocks of each size are there? Restrict attention to one eigenvalue \(\lambda\) and let \(N = T - \lambda I\). On a block \(J_k(\lambda)\), the matrix \(N_k^j\) has rank \(\max(k - j, 0)\): each power of the shift loses one dimension of image until it reaches zero. Summing over blocks, \(\operatorname{rank}(N^{j-1}) - \operatorname{rank}(N^j)\) counts the blocks for \(\lambda\) of size at least \(j\). Hence

Because ranks of \((T - \lambda I)^j\) are similarity invariants, so are these counts. Two matrices with the same eigenvalues and the same ranks of all these powers have the same Jordan form and are similar; conversely similar matrices obviously share these ranks. This proves uniqueness and gives a practical recipe. Two useful readings: the number of blocks for \(\lambda\) is \(n - r_1 = \dim E_\lambda\), the geometric multiplicity; and the size of the largest block for \(\lambda\) is the exponent of \((x - \lambda)\) in the minimal polynomial.

For small matrices the first three rows often decide the answer. A \(3 \times 3\) matrix with a single eigenvalue has Jordan form determined by its geometric multiplicity: 3 means \(\lambda I\), 2 means blocks of sizes 2 and 1, and 1 means a single block of size 3. In size 4, an eigenvalue of multiplicity 4 with geometric multiplicity 2 could have blocks 3 and 1 or 2 and 2; the minimal polynomial tells them apart.

Powers and exponentials of Jordan blocks

Since \(\lambda I\) commutes with \(N\), the binomial theorem applies, and it stops after \(k\) terms because \(N^k = 0\):

For a block of size 2, \(J^m = \begin{pmatrix} \lambda^m & m\lambda^{m-1} \\ 0 & \lambda^m \end{pmatrix}\) and \(e^{tJ} = e^{\lambda t}\begin{pmatrix} 1 & t \\ 0 & 1 \end{pmatrix}\). This is where the extra solutions \(te^{\lambda t}\) of differential equations with repeated characteristic roots come from, and the terms \(m r^m\) in recurrences with repeated roots. It also explains a subtle stability fact: if \(|\lambda| < 1\), powers \(J^m\) still go to 0, but the factor \(m\) (or \(m^{k-1}\) for larger blocks) can make them grow for a long time first. Transient growth of this kind matters in fluid dynamics and in the training of recurrent neural networks.

Why numerical software avoids the Jordan form

The Jordan form is a discontinuous function of the matrix. The matrix \(\begin{pmatrix} 0 & 1 \\ \varepsilon & 0 \end{pmatrix}\) has eigenvalues \(\pm\sqrt\varepsilon\), distinct for every \(\varepsilon > 0\), so its Jordan form is diagonal; at \(\varepsilon = 0\) it is a single block. An error of \(10^{-16}\) in one entry, the size of a rounding error, moves the eigenvalues by \(10^{-8}\). For a block of size \(k\) the eigenvalues move by about \(\varepsilon^{1/k}\). In floating point arithmetic it is therefore impossible to decide reliably whether a matrix is defective. The code shows this sensitivity directly, and then computes a Jordan structure exactly with rational arithmetic, where no such problem arises.

import numpy as np
from fractions import Fraction as F
np.set_printoptions(precision=4, suppress=True)

# sensitivity: perturb one corner of a Jordan block of size k
for k in (2, 4, 8):
    J = np.diag(np.ones(k - 1), 1)          # nilpotent block, all eigenvalues 0
    J[-1, 0] = 1e-12
    spread = max(abs(np.linalg.eigvals(J)))
    print(f"k={k}: perturbation 1e-12 moves eigenvalues to radius {spread:.2e}  (eps^(1/k) = {1e-12 ** (1 / k):.2e})")

# exact Jordan structure from ranks of powers of (A - lambda I)
def rank(M):
    M = [r[:] for r in M]
    rk = 0
    for c in range(len(M[0])):
        p = next((i for i in range(rk, len(M)) if M[i][c] != 0), None)
        if p is None:
            continue
        M[rk], M[p] = M[p], M[rk]
        for i in range(len(M)):
            if i != rk and M[i][c] != 0:
                f = M[i][c] / M[rk][c]
                M[i] = [a - f * b for a, b in zip(M[i], M[rk])]
        rk += 1
    return rk

def mul(X, Y):
    return [[sum(X[i][t] * Y[t][j] for t in range(len(Y))) for j in range(len(Y[0]))] for i in range(len(X))]

# A = P J P^(-1) for J = J_2(2) + J_1(2) + J_1(3), disguised by an integer change of basis
A = [[F(x) for x in row] for row in [[3, 0, -1, 1], [0, 2, 0, 0], [2, -1, 0, 3], [1, -1, -1, 4]]]
n = len(A)
for lam in (2, 3):
    N = [[A[i][j] - lam * (i == j) for j in range(n)] for i in range(n)]
    ranks, Pw = [n], [[F(int(i == j)) for j in range(n)] for i in range(n)]
    for _ in range(n):
        Pw = mul(Pw, N)
        ranks.append(rank(Pw))
    sizes = {j: ranks[j - 1] - 2 * ranks[j] + ranks[j + 1] for j in range(1, n) if ranks[j - 1] - 2 * ranks[j] + ranks[j + 1] > 0}
    print(f"lambda={lam}: ranks of powers {ranks[:4]}, blocks by size {sizes}")

k=2: perturbation 1e-12 moves eigenvalues to radius 1.00e-06  (eps^(1/k) = 1.00e-06)
k=4: perturbation 1e-12 moves eigenvalues to radius 1.00e-03  (eps^(1/k) = 1.00e-03)
k=8: perturbation 1e-12 moves eigenvalues to radius 3.16e-02  (eps^(1/k) = 3.16e-02)
lambda=2: ranks of powers [4, 2, 1, 1], blocks by size {1: 1, 2: 1}
lambda=3: ranks of powers [4, 3, 3, 3], blocks by size {1: 1}

For a block of size 8, a perturbation of \(10^{-12}\) moves the eigenvalues out to radius about 0.03, ten orders of magnitude larger than the perturbation. That is why libraries offer the Schur decomposition and the singular value decomposition, which are stable, and why the Jordan form is a theoretical tool and a tool for exact computation. In exact arithmetic it is perfectly computable: the ranks above identify the structure of the disguised matrix without any floating point at all.