Linear combinations
What is linear combinations? When we study linear algebra, we keep running into the idea of “linearity” in a few core ways. Each one focuses on a different kind of object:
- Linear combination (vector): We scale vectors and add them together to produce another vector. We call the resulting vector a linear combination of the original vectors.
- And linear equation is also actually asking a question about vectors: “Can we find a specific linear combination of these vectors that equals a target vector?”
- Linear independence (sets of vectors): Here, we look at a set of vectors and ask whether any vector is redundant. If one vector can be made from the others using a linear combination, we call the set linearly dependent. If no vector in the set can be produced from the others in that way, we call the set linearly independent.
- Linear transformation (functions): This time the object is a function that maps vectors from one space to another. If the function respects scaling and addition in the natural way, we call it a linear transformation.
Today, we’ll start with linear combinations.
What is a Linear Combination?
Let’s say we have a vector space \(V\) over field \(F\) and a subset \(S\) of vectors inside it. A vector \(v \in V\) is called a linear combination of vectors from \(S\) if we can build \(v\) by scaling some vectors in \(S\) and then adding them together.
Formally, if we take vectors
$$
u_1, u_2, \dots, u_n \in S
$$
and scalars
$$
a_1, a_2, \dots, a_n \in F
$$
then
$$
v = a_1u_1 + a_2u_2 + \cdots + a_nu_n
$$
is called a linear combination of the vectors \(u_1, u_2, \dots, u_n\).
The numbers \(a_1, a_2, \dots, a_n\) are called the coefficients of the linear combination.
Example 1: Linear combinations
- Vector Space: \(V = \mathbb{R}^2\) (the xy-plane)
- Field: \(F = \mathbb{R}\)
- Subset: \(S = \{ (1, 0), (0, 1) \}\)
Task:
Show that the vector \(v = (2, 5)\) is a linear combination of vectors in \(S\).
Solution:
We need to find scalars to scale \((1, 0)\) and \((0, 1)\) so they add up to \((2, 5)\).
$$
2(1, 0) + 5(0, 1) = (2, 0) + (0, 5) = (2, 5)
$$
Since we found coefficients (\(2\) and \(5\)) that work, the vector \((2, 5)\) is a linear combination of \(S\).
An Important Observation
Notice something interesting: in any vector space, multiplying any vector \(v\) by zero gives you the zero vector, \(0v = 0\). This means the zero vector is always a linear combination of any nonempty set of vectors — you just pick all coefficients equal to zero.
Span
Definition
Let \(S\) be a subset of a vector space \(V\) over a field \(F\). The span of \(S\), denoted by \(\text{span}(S)\), is the set of all possible linear combinations of the vectors in \(S\).
For convenience, we define \(\text{span}(\emptyset)=\{0\}\).
Example 2: Span
Context:
- Vector Space: \(V = \mathbb{R}^2\)
- Field: \(F = \mathbb{R}\)
- Subset: \(S = \{ (1, 0), (0, 1) \}\)
Task:
What is \(\text{span}(S)\)?
Solution:
The span is the entire vector space \(\mathbb{R}^2\).
We can confirm this because any arbitrary vector \((x, y)\) can be formed using \(S\):
$$
x(1, 0) + y(0, 1) = (x, y)
$$
Since every possible vector in the plane can be created, \(\text{span}(S) = \mathbb{R}^2\).
Example 3: Span
Context:
- Vector Space: \(V = \mathbb{R}^2\)
- Field: \(F = \mathbb{R}\)
- Subset: \(S = \{ (1, 0), (0, 1), (3, 5) \}\)
Task:
What is \(\text{span}(S)\)?
Solution:
The span is still \(\mathbb{R}^2\).
Even though we added an extra vector, we can still form any arbitrary vector \((x, y)\) by using the first two and ignoring the third:
$$
x(1, 0) + y(0, 1) + 0(3, 5) = (x, y)
$$
Adding vectors to a set never shrinks the span.
Example 4: Span
Context:
- Vector Space: \(V = \mathbb{R}^2\)
- Field: \(F = \mathbb{R}\)
- Subset: \(S = \{ (2, 4)\}\)
Task:
What is \(\text{span}(S)\)?
Solution:
\(\text{span}(S)=\{(2c, 4c)\mid c\in\mathbb{R}\}\),
which is the line through the origin with slope \(2\).
Definition of “Generate”
In Examples 2 and 3 above, we found that:
$$
\text{span}(S) = V
$$
When this equality holds—meaning every vector in the space \(V\) is included in the span of \(S\)—we say that the subset \(S\) generates (or spans) the vector space \(V\).
In this context, we also refer to the vectors in \(S\) collectively as a generating set for \(V\).
Example 5: Generating Matrices
Context:
- Vector Space: \(V = M_{2 \times 2}(\mathbb{R})\) (The set of all \(2 \times 2\) real matrices)
- Field: \(F = \mathbb{R}\)
- Subset: \(S = \left\{ \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}, \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}, \begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix}, \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix} \right\}\)
Task:
Does the subset \(S\) generate the vector space \(V\)?
Solution:
Yes. We need to check if any arbitrary matrix in \(V\) can be formed using a linear combination of the matrices in \(S\).
Let \(A = \begin{pmatrix} a & b \ c & d \end{pmatrix}\) be any matrix in \(V\). We can write it as:
$$
a\begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix} + b\begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix} + c\begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix} + d\begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} a & b \\ c & d \end{pmatrix}
$$
Since we can find coefficients (\(a, b, c, d\)) to build any matrix in the space, this subset generates the vector space \(V\).