Rank and Rank Theorem - Comprehensive Summary
Rank of Matrix: The dimension of the column space of A, equal to the number of linearly independent columns
Nullity of Matrix: The dimension of the null space of A, equal to the number of free variables in the homogeneous system
Rank Theorem: For any \(m \times n\) matrix A: \(\text{rank}(A) + \text{nullity}(A) = n\)
Column Space: The span of the column vectors of A
Null Space: The set of all solutions to \(A\vec{x} = \vec{0}\)
• Row operations don't change the rank of a matrix
• Row rank equals column rank
• Rank is the number of pivots in row-reduced form
• Nullity is the number of free variables in row-reduced form
• For square matrices: invertible ⟺ rank = dimension
1. Find rank: Row reduce to echelon form and count pivots
2. Find nullity: Count free variables in row-reduced form
3. Verify rank theorem: Check that rank + nullity = number of columns
4. Alternative: Use column space and null space dimensions
• Always verify your answer using the rank theorem
• Count pivots in row-reduced form for rank
• Free variables correspond to non-pivot columns
• Use the fundamental theorem as a verification tool
Rank of a Matrix: The rank of an \(m \times n\) matrix A is the dimension of its column space, which equals the number of linearly independent columns. Equivalently, it's the number of linearly independent rows.
- Row reduce the matrix to echelon form
- Count the number of pivot positions
- The number of pivots equals the rank
- Verify using the rank theorem if needed
\(A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 2 & 5 \end{pmatrix}\)
\(R_2 \leftarrow R_2 - 2R_1\): \(\begin{pmatrix} 1 & 2 & 3 \\ 0 & 0 & 0 \\ 1 & 2 & 5 \end{pmatrix}\)
\(R_3 \leftarrow R_3 - R_1\): \(\begin{pmatrix} 1 & 2 & 3 \\ 0 & 0 & 0 \\ 0 & 0 & 2 \end{pmatrix}\)
\(R_3 \leftrightarrow R_2\): \(\begin{pmatrix} 1 & 2 & 3 \\ 0 & 0 & 2 \\ 0 & 0 & 0 \end{pmatrix}\)
\(R_2 \leftarrow \frac{1}{2}R_2\): \(\begin{pmatrix} 1 & 2 & 3 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}\)
\(R_1 \leftarrow R_1 - 3R_2\): \(\begin{pmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}\)
In the row-reduced form \(\begin{pmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}\):
Pivot 1: Position (1,1) - leading 1 in row 1
Pivot 2: Position (2,3) - leading 1 in row 2
Total: 2 pivot positions
The rank equals the number of pivot positions.
Therefore: \(\text{rank}(A) = 2\)
Original columns: \(\vec{c_1} = \begin{pmatrix} 1 \\ 2 \\ 1 \end{pmatrix}\), \(\vec{c_2} = \begin{pmatrix} 2 \\ 4 \\ 2 \end{pmatrix}\), \(\vec{c_3} = \begin{pmatrix} 3 \\ 6 \\ 5 \end{pmatrix}\)
Notice: \(\vec{c_2} = 2\vec{c_1}\), so columns 1 and 2 are linearly dependent.
Also, \(\vec{c_3}\) is not a scalar multiple of \(\vec{c_1}\) (check: if \(\vec{c_3} = k\vec{c_1}\), then \(3 = k\), but then \(5 \neq k\cdot 1\)).
So \(\{\vec{c_1}, \vec{c_3}\}\) is a linearly independent set that spans the column space.
Dimension of column space = 2 = rank. ✓
The rank of A is 2.
• Row operations preserve rank: Elementary row operations don't change the rank
• Pivot counting: Number of pivots equals rank
• Linear independence: Count independent column vectors
Nullity of a Matrix: The nullity of an \(m \times n\) matrix A is the dimension of its null space, which equals the number of free variables in the homogeneous system \(A\vec{x} = \vec{0}\).
\(\begin{pmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}\)
From the row-reduced form: \(x_1 + 2x_2 = 0\) and \(x_3 = 0\)
Pivot variables (corresponding to pivot columns): \(x_1, x_3\)
Free variable: \(x_2\) (no pivot in column 2)
From \(x_1 + 2x_2 = 0\): \(x_1 = -2x_2\)
From \(x_3 = 0\): \(x_3 = 0\)
Let \(x_2 = t\) where \(t\) is any real number.
Then: \(\begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \begin{pmatrix} -2t \\ t \\ 0 \end{pmatrix} = t\begin{pmatrix} -2 \\ 1 \\ 0 \end{pmatrix}\)
\(\text{null}(A) = \text{span}\left\{\begin{pmatrix} -2 \\ 1 \\ 0 \end{pmatrix}\right\}\)
Since the null space has a basis with 1 vector, \(\text{nullity}(A) = 1\)
Matrix A is \(3 \times 3\), so \(n = 3\).
\(\text{rank}(A) + \text{nullity}(A) = 2 + 1 = 3 = n\). ✓
Number of free variables = Total variables - Number of pivots = 3 - 2 = 1
Therefore, \(\text{nullity}(A) = 1\). ✓
The nullity of A is 1, and the Rank Theorem is verified: \(\text{rank}(A) + \text{nullity}(A) = 2 + 1 = 3\).
• Free variable counting: Nullity equals number of free variables
• Rank-Nullity theorem: Fundamental relationship for verification
• Dimension calculation: Count vectors in null space basis
Full Rank Matrix: A matrix is said to have full rank if its rank equals the minimum of its number of rows and columns. For an \(n \times n\) square matrix, full rank means \(\text{rank} = n\).
\(B = \begin{pmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \end{pmatrix}\) is already in row echelon form.
In the matrix \(\begin{pmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \end{pmatrix}\):
Pivot 1: Position (1,1) - leading 1 in row 1
Pivot 2: Position (2,2) - leading 1 in row 2
Pivot 3: Position (3,3) - leading 1 in row 3
Total: 3 pivot positions
Since there are 3 pivot positions, \(\text{rank}(B) = 3\).
Since every column has a pivot, there are no free variables.
Therefore: \(\text{nullity}(B) = 0\)
Matrix B is \(3 \times 3\), so \(n = 3\).
\(\text{rank}(B) + \text{nullity}(B) = 3 + 0 = 3 = n\). ✓
Solve \(B\vec{x} = \vec{0}\): \(\begin{pmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \end{pmatrix}\begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}\)
This gives us: \(x_1 + 2x_3 = 0\), \(x_2 + 3x_3 = 0\), \(x_3 = 0\)
From the third equation: \(x_3 = 0\)
Substituting into the first: \(x_1 = 0\)
Substituting into the second: \(x_2 = 0\)
Only solution: \(\vec{x} = \vec{0}\), so null space is \(\{\vec{0}\}\) and nullity = 0. ✓
The rank of B is 3 and the nullity of B is 0. Matrix B has full rank.
• Full rank matrix: All columns are linearly independent
• Trivial null space: Full rank ⟹ nullity = 0
• Triangular form: Easy to identify pivots in triangular matrices
Rank of Product: For matrices A and B where the product AB is defined, \(\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))\). This is because the column space of AB is contained in the column space of A.
For matrices A (size \(m \times n\)) and B (size \(n \times p\)), the product AB is defined and has size \(m \times p\).
Here: A is \(3 \times 4\) and B is \(4 \times 2\), so AB is \(3 \times 2\).
The fundamental inequality for matrix ranks states:
\(\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))\)
This is because the column space of AB is contained in the column space of A.
Also, the row space of AB is contained in the row space of B.
Given: \(\text{rank}(A) = 2\) and \(\text{rank}(B) = 2\)
Therefore: \(\text{rank}(AB) \leq \min(2, 2) = 2\)
It's possible to construct examples where \(\text{rank}(AB) = 2\).
For example, if A and B are chosen such that the image of B is not in the null space of A, then the maximum rank can be achieved.
Since AB is \(3 \times 2\), the maximum possible rank is \(\min(3, 2) = 2\).
This agrees with our bound from the rank inequality.
The maximum possible rank of AB is 2.
• Rank inequality: \(\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))\)
• Dimension constraint: Rank ≤ min(rows, cols)
• Subspace containment: Col(AB) ⊆ Col(A) and Row(AB) ⊆ Row(B)
Rank Bounds: The rank of an \(m \times n\) matrix A is bounded by the number of rows and columns. The rank equals the dimension of the column space, which is a subspace of \(\mathbb{R}^m\), and also equals the dimension of the row space, which is a subspace of \(\mathbb{R}^n\).
The rank of A equals the dimension of the column space of A.
The column space of A consists of all linear combinations of the columns of A.
Since each column of A has m entries, each column is in \(\mathbb{R}^m\).
Therefore, the column space is a subspace of \(\mathbb{R}^m\).
Hence: \(\text{rank}(A) = \dim(\text{col}(A)) \leq \dim(\mathbb{R}^m) = m\)
The rank of A also equals the dimension of the row space of A.
The row space of A consists of all linear combinations of the rows of A.
Since A has n columns, each row has n entries, so each row is in \(\mathbb{R}^n\).
Therefore, the row space is a subspace of \(\mathbb{R}^n\).
Hence: \(\text{rank}(A) = \dim(\text{row}(A)) \leq \dim(\mathbb{R}^n) = n\)
From Steps 1 and 2: \(\text{rank}(A) \leq m\) and \(\text{rank}(A) \leq n\)
Therefore: \(\text{rank}(A) \leq \min(m, n)\)
Equality \(\text{rank}(A) = \min(m, n)\) occurs when A has full rank.
Case 1: If \(m \geq n\), then \(\text{rank}(A) = n\) means A has full column rank (columns are linearly independent).
Case 2: If \(m \leq n\), then \(\text{rank}(A) = m\) means A has full row rank (rows are linearly independent).
Case 3: If \(m = n\), then \(\text{rank}(A) = n\) means A has full rank (both row and column rank are full), which is equivalent to A being invertible.
Example where \(\text{rank}(A) = \min(m,n)\): For a \(3 \times 2\) matrix with 2 linearly independent columns, \(\text{rank} = 2 = \min(3,2)\).
Example where \(\text{rank}(A) < \min(m,n)\): For a \(3 \times 3\) matrix with rank 2, \(\text{rank} = 2 < 3 = \min(3,3)\).
For any \(m \times n\) matrix A, \(\text{rank}(A) \leq \min(m, n)\). Equality occurs when A has full rank: full column rank if \(m \geq n\), or full row rank if \(m \leq n\).
• Subspace dimension: Dimension of subspace ≤ dimension of space
• Column-row rank equality: Row rank equals column rank
• Full rank characterization: When rank achieves maximum possible value
Rank: The dimension of the column space (or equivalently, the row space) of a matrix. It measures the maximum number of linearly independent rows or columns.
Nullity: The dimension of the null space of a matrix. It measures the number of degrees of freedom in the solution space of the homogeneous system.
Rank Theorem: For any \(m \times n\) matrix A: \(\text{rank}(A) + \text{nullity}(A) = n\).
- For rank: Row reduce to echelon form and count pivot positions
- For nullity: Count free variables or use rank-nullity theorem
- For verification: Use rank theorem as a consistency check
- For applications: Use rank to determine invertibility, consistency, etc.
• \(\text{rank}(A) = \text{rank}(A^T)\) (row rank equals column rank)
• \(\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))\)
• \(A\) is invertible ⟺ \(\text{rank}(A) = n\) for \(n \times n\) matrix
• Homogeneous system \(A\vec{x} = \vec{0}\) has non-trivial solutions ⟺ \(\text{nullity}(A) > 0\)
• \(\text{rank}(A) \leq \min(m,n)\) for \(m \times n\) matrix A
Geometric Interpretation: A matrix with rank r maps vectors to an r-dimensional subspace. For a rank-2 matrix, all outputs lie in a 2D plane within the codomain.
- Input space: \(\mathbb{R}^3\) (3D)
- Matrix rank: 2
- Output space: 2D subspace of codomain
- Null space: 1D (by rank theorem: 3-2=1)