07 | Vector spaces associated with matrices
07 | Vector spaces associated with matrices
This chapter of Linear Algebra by Dr JH Klopper is licensed under an Attribution-NonCommercial-NoDerivatives 4.0 International Licence available at http://creativecommons.org/licenses/by-nc-nd/4.0/?ref=chooser-v1 .
7.1 Introduction
7.1 Introduction
In this notebook, we investigate the very important vector spaces of matrices called the column space, the row space, and the null space.
If we consider all matrices, , then the vector space spanned by all the column vectors in is the column space of . The vector space spanned by the row vectors of is the row space of . The solution space of the homogeneous linear system is the null space of . We explore these concepts in this chapter.
m×n
A
A
A
A
A
Ax=0
A
7.2 Column space of a matrix
7.2 Column space of a matrix
Consider the linear system in (1), where , matrices with rows and columns of real numbers.
A∈
m×n
m
n
Ax=b
=
a 11 | a 12 | ⋯ | a 1n |
a 21 | a 22 | ⋯ | a 2n |
⋮ | ⋮ | ⋱ | ⋮ |
a m1 | a m2 | ⋯ | a mn |
m×n
x 1 |
x 2 |
⋮ |
x n |
n×1
b 1 |
b 2 |
⋮ |
b m |
m×1
(
1
)Definition 7.2.1 The space spanned by the columns of a matrix is the column space of .
A
A
We note that is in space. Hence a linear combination of the columns (column vectors) in spans at most . This will only happen if the column vectors in are linearly independent. If so, the system is consistent as is in the column space of . In fact, the linear system is only consistent if is in the column space of .
b
m
A
m
A
b
A
Ax=b
b
A
Since there are at most column vectors in , the dimension of the space that it spans is at most . Remember that the rank, , of a matrix is the number of pivots. If the rank equals the dimension of (here ) then spans .
n
A
n
r
b
m
A
m
In (2), we look at an example to demonstrate how the rank of a matrix determines the dimension of the column space that it spans.
A=
1 | 3 | 4 |
2 | 1 | 3 |
1 | 1 | 0 |
2 | 0 | 3 |
4×3
(
2
)We create the matrix below as a list-of lists and assign it to the variable A.
A={{1,3,4},{2,1,3},{1,1,0},{2,0,3}};MatrixForm[A](*ShowingthematrixformofA*)
Out[]//MatrixForm=
1 | 3 | 4 |
2 | 1 | 3 |
1 | 1 | 0 |
2 | 0 | 3 |
Since this is a matrix, we can only multiply it with a vector. The results is a vector, . Do the three column vectors in span ? Gauss-Jordan elimination shows only pivots as expected.
4×3
3×1
4×1
b∈
4
A
4
3
In[]:=
MatrixForm[RowReduce[A]]
Out[]//MatrixForm=
1 | 0 | 0 |
0 | 1 | 0 |
0 | 0 | 1 |
0 | 0 | 0 |
The MatrixRank function confirms that the rank is only .
3
In[]:=
MatrixRank[A]
Out[]=
3
The augmented matrix of the homogeneous system shows that the column vectors are independent.
In[]:=
MatrixForm[RowReduce[{{1,3,4,0},{2,1,3,0},{1,1,0,0},{2,0,3,0}}]]
Out[]//MatrixForm=
1 | 0 | 0 | 0 |
0 | 1 | 0 | 0 |
0 | 0 | 1 | 0 |
0 | 0 | 0 | 0 |
The column space of is only . This is a subspace of .
A
3
4
In the next example, we have another matrix, .
A
3×4
A=
,x=
,b=
1 | 2 | 3 | 1 |
3 | 3 | 4 | 1 |
2 | 1 | 1 | -1 |
3×4
x 1 |
x 2 |
x 3 |
x 4 |
4×1
b 1 |
b 2 |
b 3 |
3×1
(
3
)We create and assign it to the variable A.
A
In[]:=
A={{1,2,3,1},{3,3,4,1},{2,1,1,-1}};MatrixForm[A]
Out[]//MatrixForm=
1 | 2 | 3 | 1 |
3 | 3 | 4 | 1 |
2 | 1 | 1 | -1 |
We have four column vectors in and . The four column vectors are each in . Since we need at most a set of three independent vectors as basis for the set, we do not have linear independence. Let’s look at the rank of .
A
b∈
3
3
A
In[]:=
MatrixRank[A]
Out[]=
3
Gauss-Jordan elimination shows pivots.
3
In[]:=
MatrixForm[RowReduce[A]]
Out[]//MatrixForm=
1 | 0 | - 1 3 | 0 |
0 | 1 | 5 3 | 0 |
0 | 0 | 0 | 1 |
We can remove one of the column vectors, say the last one, and see if the we have three independent vectors. Instead of calculating if the homogeneous system only has the zero vector as solution, we use the determinant.
In[]:=
Det[{{1,2,3},{3,3,4},{2,1,1}}]
Out[]=
0
That did not work. Let’s drop the second last column vector.
In[]:=
Det[{{1,3,1},{3,3,1},{2,1,-1}}]
Out[]=
8
These three remaining column vectors are a basis for and any will be in the column space of .
3
b∈
3
A
We state that a linear system is consistent if and only if is in the column space of .
Ax=b
b
A
7.3 Nullspace
7.3 Nullspace
The null space is the space of vectors such that . As an example, we consider a matrix with three column vectors, each in .
x
Ax=0
3
In[]:=
A={{1,2,3},{2,3,4},{4,5,1}};MatrixForm[A]
Out[]//MatrixForm=
1 | 2 | 3 |
2 | 3 | 4 |
4 | 5 | 1 |
The three column vectors are linearly independent as we can see from the non-zero determinant.
In[]:=
Det[A]
Out[]=
5
The null space is now the space spanned by all the vectors such that . Let’s look at the reduced row-echelon form of the augmented matrix representing the homogeneous linear system. We use the MapThread and Append functions to add a zero-vector as the last column.
x
Ax=0
In[]:=
MatrixForm[MapThread[Append,{A,{0,0,0}}]]
Out[]//MatrixForm=
1 | 2 | 3 | 0 |
2 | 3 | 4 | 0 |
4 | 5 | 1 | 0 |
In[]:=
MatrixForm[RowReduce[MapThread[Append,{A,{0,0,0}}]]]
Out[]//MatrixForm=
1 | 0 | 0 | 0 |
0 | 1 | 0 | 0 |
0 | 0 | 1 | 0 |
The null space of matrix is the vector space that only contains the zero-vector in .
A
3
In the next example, we have three column vectors, each in . By necessity, and . We are only interested in the homogeneous system when considering the nullspace, though.
4
x∈
3
b∈
4
In[]:=
A={{1,2,3},{2,3,4},{4,5,1},{1,0,5}};MatrixForm[A]
Out[]//MatrixForm=
1 | 2 | 3 |
2 | 3 | 4 |
4 | 5 | 1 |
1 | 0 | 5 |
Let’s look at the reduced row-echelon form of the augmented matrix.
In[]:=
MatrixForm[RowReduce[MapThread[Append,{A,{0,0,0,0}}]]]
Out[]//MatrixForm=
1 | 0 | 0 | 0 |
0 | 1 | 0 | 0 |
0 | 0 | 1 | 0 |
0 | 0 | 0 | 0 |
Again we find that the nullspace of contains only the zero vector.
A
Now for an example with a non-trivial nullspace. We have three column vectors, each in . Hence, . We solve the homogeneous linear system.
3
x∈
3
In[]:=
A={{1,2,3},{2,3,5},{4,5,9}};MatrixForm[A]
Out[]//MatrixForm=
1 | 2 | 3 |
2 | 3 | 5 |
4 | 5 | 9 |
In[]:=
MatrixForm[RowReduce[MapThread[Append,{A,{0,0,0}}]]]
Out[]//MatrixForm=
1 | 0 | 1 | 0 |
0 | 1 | 1 | 0 |
0 | 0 | 0 | 0 |
The solution is shown in (4).
The nullspace is calculated below and shown in (5).
The NullSpace function returns a nested array of the nullspace vectors. We use it to check our work.
This brings us to a very important theorem.
7.4 Row space
7.4 Row space
We can also consider the rows in a matrix as row vectors.
The three row vectors are shown in (10).
7.5 Basis for column space, row space, and nullspace
7.5 Basis for column space, row space, and nullspace
Note that we we able to use these arguments, because elementary row operations to not change the row of column space of a matrix.
7.6 Rank
7.6 Rank
Notice the following.