Subspaces, rank and the structure of solutions · MATH-125 Lesson 7

Lesson 3 found that a linear system can have exactly one solution, none, or infinitely many, and Lesson 6 showed that a square system has exactly one solution when the determinant of its matrix is not zero. Those facts answer the question "is there a solution?" but not the deeper questions an engineer often needs answered. If a system has no solution, which right-hand sides can it reach? If it has infinitely many, what freedom do the solutions have? How many independent equations does a model really contain? This lesson answers these questions with a few central ideas: subspaces, the column space and null space of a matrix, its rank, and the rank-nullity theorem that ties them together. They explain, among other things, why some trusses are rigid and others collapse, why some measurement sets determine a position and others do not, and how many independent loop equations a circuit has.

Subspaces

A subspace of $\mathbb{R}^n$ is a set of vectors that contains the zero vector and is closed under addition and scalar multiplication: adding two vectors in the set, or scaling one, gives another vector in the set. Geometrically, the subspaces of $\mathbb{R}^3$ are the origin alone, lines through the origin, planes through the origin, and all of $\mathbb{R}^3$. Every span is a subspace (Lesson 2), and every subspace is the span of some set of vectors.

A line or plane that does not pass through the origin is not a subspace, since it does not contain the zero vector. Such sets appear as the solution sets of systems with non-zero right-hand sides, and, as shown below, they are subspaces shifted away from the origin.

Every subspace has a basis, a linearly independent set that spans it, and every basis of a given subspace has the same number of vectors, its dimension. A line through the origin has dimension 1, a plane through the origin dimension 2.

The column space

The product $A\mathbf{x}$ is a linear combination of the columns of $A$, with the entries of $\mathbf{x}$ as weights (Lesson 4). So the system $A\mathbf{x} = \mathbf{b}$ has a solution exactly when $\mathbf{b}$ is a linear combination of the columns. The set of all such $\mathbf{b}$ is the column space of $A$, written $\operatorname{Col}(A)$: the span of the columns, everything the matrix can produce.

For an $m \times n$ matrix, the column space is a subspace of $\mathbb{R}^m$. If it is all of $\mathbb{R}^m$, every right-hand side can be reached. If it is smaller, some right-hand sides are impossible, and the system is inconsistent for them.

Worked problem

Which right-hand sides work?

For $A = \begin{pmatrix} 1 & 2 \\ 2 & 4 \\ 3 & 6 \end{pmatrix}$, describe the column space and decide whether $A\mathbf{x} = (1, 2, 3)$ and $A\mathbf{x} = (1, 2, 4)$ have solutions.

Show solution

The second column is twice the first, so the column space is the line spanned by $(1, 2, 3)$.
$(1, 2, 3)$ lies on this line: $\mathbf{x} = (1, 0)$ works, as does $(-1, 1)$ and many others.
$(1, 2, 4)$ is not a multiple of $(1, 2, 3)$, so there is no solution.

Answer: The column space is a line in ℝ³; only right-hand sides on that line can be reached, and for those there are infinitely many solutions.

The null space

The null space of $A$, written $\operatorname{Null}(A)$, is the set of all solutions of the homogeneous system $A\mathbf{x} = \mathbf{0}$. It is a subspace of $\mathbb{R}^n$, because sums and multiples of solutions of a homogeneous system are again solutions. It always contains $\mathbf{x} = \mathbf{0}$; the question is whether it contains anything else.

To find a basis for the null space, reduce $A$ to reduced row echelon form (Lesson 3), identify the free variables (columns without pivots), and write the general solution as a combination of vectors, one for each free variable. Those vectors form a basis.

Worked problem

A null space basis

Find a basis for the null space of $A = \begin{pmatrix} 1 & 2 & 0 & 3 \\ 0 & 0 & 1 & -1 \end{pmatrix}$, which is already in reduced form.

Show solution

Pivots in columns 1 and 3; free variables $x_2 = s$, $x_4 = t$.
Equations: $x_1 = -2s - 3t$, $x_3 = t$.
General solution: $\mathbf{x} = s(-2, 1, 0, 0) + t(-3, 0, 1, 1)$.

Answer: The null space is the plane in ℝ⁴ spanned by (−2, 1, 0, 0) and (−3, 0, 1, 1); its dimension is 2, the number of free variables.

The structure of all solutions

The null space describes the freedom in every solution of $A\mathbf{x} = \mathbf{b}$. If $\mathbf{x}_p$ is any one particular solution, then every solution has the form:

\[\mathbf{x} = \mathbf{x}_p + \mathbf{x}_n, \qquad \mathbf{x}_n \in \operatorname{Null}(A)\]

because $A(\mathbf{x}_p + \mathbf{x}_n) = \mathbf{b} + \mathbf{0} = \mathbf{b}$, and the difference of any two solutions satisfies $A\mathbf{x} = \mathbf{0}$. The solution set is therefore the null space shifted to pass through $\mathbf{x}_p$: a point if the null space is just $\{\mathbf{0}\}$, a line, a plane, and so on. The same structure, particular solution plus homogeneous solution, governs linear differential equations (MATH-201), where the homogeneous solutions describe free vibrations or transients and the particular solution the forced response.

$Diagram in the plane showing the null space of a matrix as a line through the origin, labeled null space, and a parallel line shifted away from the origin passing through a point x p, labeled all solutions of A x equals b. An arrow from the origin to x p and an arrow along the shifted line indicate that every solution is x p plus a vector in the null space.$

The solutions of Ax = b form the null space shifted by a particular solution.

Rank

The rank of a matrix is the number of pivots in its echelon form. It equals the dimension of the column space (the pivot columns of the original matrix form a basis for it), and, remarkably, also the dimension of the row space, the span of the rows. The rank therefore counts the number of genuinely independent equations in a system: if one equation is a combination of others, it adds nothing, and elimination reduces it to a row of zeros.

Rank also answers existence and uniqueness questions concisely:

$A\mathbf{x} = \mathbf{b}$ has a solution if and only if $\operatorname{rank}(A) = \operatorname{rank}([A \mid \mathbf{b}])$: adding $\mathbf{b}$ as an extra column does not create a new pivot.
If a solution exists, it is unique if and only if $\operatorname{rank}(A) = n$, the number of unknowns (no free variables).
A square $n \times n$ matrix is invertible if and only if its rank is $n$, equivalently its determinant is non-zero (Lesson 6).

The rank-nullity theorem

Each column of an $m \times n$ matrix either contains a pivot or corresponds to a free variable, so the number of pivots plus the number of free variables equals the number of columns:

\[\operatorname{rank}(A) + \dim\operatorname{Null}(A) = n\]

This simple count is the rank-nullity theorem. It means that every independent constraint (pivot) removes one degree of freedom, and the freedom that remains is the dimension of the null space. A system of 5 equations in 8 unknowns with rank 5 leaves a 3-dimensional family of solutions; if the rank is only 4, because one equation is redundant, the family is 4-dimensional.

Worked problem

Applying rank-nullity

A 6 × 9 matrix has rank 5. What are the dimensions of its column space and null space? Can $A\mathbf{x} = \mathbf{b}$ be solved for every $\mathbf{b}$ in $\mathbb{R}^6$?

Show solution

Column space dimension = rank = 5.
Null space dimension = $9 - 5 = 4$.
The column space is a 5-dimensional subspace of $\mathbb{R}^6$, not all of it, so some $\mathbf{b}$ cannot be reached.

Answer: Column space of dimension 5, null space of dimension 4; not every right-hand side is reachable, and when one is, the solutions form a 4-dimensional family.

Engineering applications

Trusses: rigid, unstable or indeterminate

In a planar pin-jointed truss with $j$ joints, $m$ members and $r$ support reactions, the equilibrium of each joint gives two equations, so there are $2j$ equations in $m + r$ unknown forces. Writing them as $A\mathbf{f} = \mathbf{p}$, where $\mathbf{p}$ holds the applied loads:

If $\operatorname{rank}(A) = 2j = m + r$, every load can be carried and the member forces are unique: the truss is statically determinate and stable.
If $m + r > \operatorname{rank}(A) = 2j$, every load can be carried but the forces are not unique: the null space contains self-stress states, sets of member forces in equilibrium with no load, and the truss is statically indeterminate. Its forces depend on the stiffness of the members, which is why indeterminate structures need the stiffness methods of later courses.
If $\operatorname{rank}(A) < 2j$, some loads cannot be carried: the truss is a mechanism that will move. A rank deficiency can occur even when $m + r = 2j$, if members are badly arranged, so counting members is not enough; the rank is what matters.

Worked problem

Counting a truss

A planar truss has 6 joints, 9 members, and is supported by a pin (2 reactions) and a roller (1 reaction). If its equilibrium matrix has full rank, is it determinate?

Show solution

Equations: $2j = 12$. Unknowns: $m + r = 9 + 3 = 12$.
Full rank means rank 12, equal to both counts.

Answer: Yes, statically determinate and stable: every load produces a unique set of member forces. Adding a tenth member would make it indeterminate with a one-dimensional space of self-stresses.

Circuits: how many independent equations?

In a circuit with $b$ branches and $n$ nodes (in one connected piece), Kirchhoff's current law at every node gives $n$ equations, but they are not independent: adding them all gives zero, because each branch current leaves one node and enters another. The rank of the node equations is $n - 1$, and by rank-nullity the space of current patterns satisfying the current law has dimension $b - (n - 1) = b - n + 1$. This is the number of independent loops, the number of loop equations used in mesh analysis. A circuit with 10 branches and 6 nodes has 5 independent loops.

Surveying and positioning

Determining a position from measurements is a linear (or linearized) system. If the measurements are poorly arranged, for example angles measured to targets that all lie in nearly the same direction, the matrix of the system is nearly rank deficient, and some combination of coordinates is barely determined. GPS receivers report a dilution of precision that measures exactly this effect: when visible satellites are clustered in one part of the sky, the geometry matrix is poorly conditioned and the position error grows.

Common mistakes

MistakeCalling any line or plane a subspace.
InsteadA subspace must contain the origin.
MistakeUsing the columns of the echelon form as a basis for the column space.
InsteadUse the pivot columns of the original matrix.
MistakeAssuming m equations in n unknowns have rank m.
InsteadRedundant equations reduce the rank; compute it.
MistakeJudging a truss stable by counting members alone.
InsteadCheck the rank of the equilibrium matrix; poor arrangement can create mechanisms.
MistakeWriting all n node equations of a circuit as independent.
InsteadOnly n − 1 node equations are independent.

Summary and key terms

A subspace contains the origin and is closed under addition and scaling; spans are subspaces.
The column space is everything Ax can produce; Ax = b is solvable exactly when b is in it.
The null space is the solution set of Ax = 0; a basis comes from the free variables.
All solutions of Ax = b are a particular solution plus the null space.
Rank is the number of pivots, the dimension of both the column space and the row space.
Rank-nullity: rank(A) + dim Null(A) = n.
Rank determines truss stability and determinacy, the number of independent circuit loops, and positioning geometry.

Subspace: A set of vectors containing zero, closed under addition and scalar multiplication.
Column space: The span of the columns of a matrix.
Null space: The set of solutions of Ax = 0.
Row space: The span of the rows of a matrix.
Rank: The number of pivots; the dimension of the column space.
Free variable: An unknown corresponding to a column without a pivot.
Particular solution: Any one solution of Ax = b.
Rank-nullity theorem: Rank plus null space dimension equals the number of columns.
Statically indeterminate: Having more unknown forces than independent equilibrium equations.
Mechanism: A structure that can move without deforming its members.

Practice problems

Work each problem on paper first, then reveal the answer to check.

Is the set of vectors (x, y, z) with x + y + z = 1 a subspace?
Show answer
No; it does not contain the origin.
Is the set with x + y + z = 0 a subspace? What is its dimension?
Show answer
Yes; a plane through the origin, dimension 2.
Find the rank of $\begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 0 & 1 \end{pmatrix}$.
Show answer
2.
Find a basis for the null space of $\begin{pmatrix} 1 & 1 & 1 \end{pmatrix}$.
Show answer
(−1, 1, 0) and (−1, 0, 1).
A 4 × 7 matrix has rank 4. Dimension of its null space?
Show answer
3.
For a 5 × 5 matrix of rank 5, describe the solutions of Ax = b.
Show answer
Exactly one solution for every b.
If x_p = (1, 2) solves Ax = b and Null(A) is spanned by (1, −1), give all solutions.
Show answer
(1 + t, 2 − t).
A truss has 5 joints, 7 members and 3 reactions. Is the count consistent with determinacy?
Show answer
Yes: 2j = 10 = m + r; check that the rank is 10.
A circuit has 8 branches and 5 nodes. How many independent loops?
Show answer
8 − 5 + 1 = 4.
What does a large GPS dilution of precision indicate about the geometry matrix?
Show answer
It is nearly rank deficient (poorly conditioned).