Linear combinations, span and independence

Lesson 1 treated vectors one or two at a time: adding them, scaling them, measuring the angle between them. Linear algebra becomes powerful when we ask what can be built from a collection of vectors. Which forces can three cables supply to hold a load in place? Which colours can a screen produce from red, green and blue light? Which mixtures of three alloys give a required composition? Which positions can a robot arm reach by combining joint motions? Each question asks about linear combinations, sums of scaled vectors. This lesson introduces linear combinations and the span of a set of vectors, the set of everything they can build, and the concept of linear independence, which tells us when a set of vectors has no redundancy. These ideas run through the rest of the course: solving linear systems (Lesson 3), understanding matrices (Lessons 4 and 7), and choosing good coordinates (Lessons 9 and 10).

Linear combinations

A linear combination of vectors \(\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_k\) is any vector of the form:

where the scalars \(c_1, \ldots, c_k\) are called the weights or coefficients. Geometrically, a linear combination of two vectors is reached by travelling some distance along the first and then some distance along the second, as when a ship sails partly north and partly east, or when two cables together pull a load in a direction neither pulls alone.

Many engineering questions take the form: can a given vector \(\mathbf{b}\) be written as a linear combination of given vectors, and if so, with which weights? Writing out the components turns this into a system of linear equations.

Span

The span of a set of vectors is the set of all their linear combinations, written \(\operatorname{span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\}\). It is everything that can be built from the set.

• The span of a single non-zero vector in \(\mathbb{R}^3\) is a line through the origin in its direction.
• The span of two non-parallel vectors in \(\mathbb{R}^3\) is a plane through the origin containing both.
• The span of three vectors in \(\mathbb{R}^3\) that do not lie in a common plane is all of \(\mathbb{R}^3\).

Spans always pass through the origin, since choosing all weights zero gives the zero vector. A set of vectors spans \(\mathbb{R}^n\) if every vector in \(\mathbb{R}^n\) is a linear combination of them. For example, the standard unit vectors \(\mathbf{i}, \mathbf{j}, \mathbf{k}\) span \(\mathbb{R}^3\), since \((a, b, c) = a\mathbf{i} + b\mathbf{j} + c\mathbf{k}\).

Linear independence

Sometimes a set of vectors contains redundancy: one of them can be built from the others, and removing it does not change the span. For example, \((1, 0)\), \((0, 1)\) and \((2, 3)\) span \(\mathbb{R}^2\), but so do the first two alone, since \((2, 3) = 2(1, 0) + 3(0, 1)\).

Vectors \(\mathbf{v}_1, \ldots, \mathbf{v}_k\) are linearly independent if the only way to combine them to give the zero vector is with all weights zero:

Otherwise they are linearly dependent, and at least one of them is a linear combination of the others. Some quick tests:

• Two vectors are dependent exactly when one is a multiple of the other (they are parallel).
• Any set containing the zero vector is dependent.
• Any set of more than \(n\) vectors in \(\mathbb{R}^n\) is dependent: in the plane, three vectors can never be independent.
• Three vectors in \(\mathbb{R}^3\) are independent exactly when the scalar triple product from Lesson 1, which is the determinant of the matrix with the vectors as columns (Lesson 6), is not zero, meaning they do not lie in a common plane.

Bases and coordinates

A basis for a space is a set of vectors that is linearly independent and spans the space: enough vectors to build everything, and no more. Every basis of \(\mathbb{R}^n\) has exactly \(n\) vectors, and \(n\) is called the dimension. Given a basis, every vector can be written as a linear combination of the basis vectors in exactly one way; the weights are the coordinates of the vector in that basis.

The standard basis is the most familiar, but other bases are often more convenient. On an inclined plane, it is easier to use one basis vector along the slope and one perpendicular to it. For a crystal, the natural basis follows the lattice. For a vibrating structure, the most useful basis consists of its natural mode shapes, which Lesson 9 finds as eigenvectors. Changing basis does not change a vector; it changes only the numbers used to describe it.

Engineering examples

Statics: can the cables hold the load?

A load held by several cables is in equilibrium when the cable tensions, each a positive multiple of a unit vector along its cable, sum to balance the weight. The question "can these cables support this load?" is a span question with an extra condition: the weights (tensions) must be non-negative, since cables can pull but not push. Two cables in a vertical plane can support a load hanging between them, but two cables in three-dimensional space span only a plane, so a sideways force would swing the load: at least three non-coplanar cables are needed for full three-dimensional support.

Colour mixing

A display creates colours by adding red, green and blue light in varying intensities. Each colour the eye perceives can be represented, approximately, as a vector of three numbers, and the colours a display can show are the linear combinations of its three primaries with weights between 0 and full intensity. The primaries are linearly independent, so they span a three-dimensional space of colours, but because weights cannot be negative, the reachable colours fill only a region (the display's gamut), and some very saturated colours lie outside it.

Redundant measurements

In surveying, a position can be fixed from two independent distance measurements in the plane, but surveyors take more measurements than strictly necessary. The extra measurements are dependent on the others in an ideal world, but real measurements contain errors, so they disagree slightly. Redundancy allows errors to be detected and averaged out by least squares, the subject of Lesson 10.