Solved Exercises on Rank and Rank Theorem

Master matrix rank, column space, null space, and the fundamental rank theorem through these 5 detailed exercises.

Rank and Rank Theorem - Comprehensive Summary

Definitions

Rank of Matrix: The dimension of the column space of A, equal to the number of linearly independent columns

Nullity of Matrix: The dimension of the null space of A, equal to the number of free variables in the homogeneous system

Rank Theorem: For any \(m \times n\) matrix A: \(\text{rank}(A) + \text{nullity}(A) = n\)

Column Space: The span of the column vectors of A

Null Space: The set of all solutions to \(A\vec{x} = \vec{0}\)

Core Rules

• Row operations don't change the rank of a matrix

• Row rank equals column rank

• Rank is the number of pivots in row-reduced form

• Nullity is the number of free variables in row-reduced form

• For square matrices: invertible ⟺ rank = dimension

Methods

1. Find rank: Row reduce to echelon form and count pivots

2. Find nullity: Count free variables in row-reduced form

3. Verify rank theorem: Check that rank + nullity = number of columns

4. Alternative: Use column space and null space dimensions

Key Tips

• Always verify your answer using the rank theorem

• Count pivots in row-reduced form for rank

• Free variables correspond to non-pivot columns

• Use the fundamental theorem as a verification tool

Solution: Exercises 1 to 3
1 Finding Rank by Row Reduction
Exercise 1
Find the rank of \(A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 2 & 5 \end{pmatrix}\).
Definition:

Rank of a Matrix: The rank of an \(m \times n\) matrix A is the dimension of its column space, which equals the number of linearly independent columns. Equivalently, it's the number of linearly independent rows.

Rank Calculation Method:
  1. Row reduce the matrix to echelon form
  2. Count the number of pivot positions
  3. The number of pivots equals the rank
  4. Verify using the rank theorem if needed
Original Matrix
\(A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 2 & 5 \end{pmatrix}\)
Row Reduced
\(\begin{pmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}\)
Rank
2
Step 1: Row reduce the matrix

\(A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 2 & 5 \end{pmatrix}\)

\(R_2 \leftarrow R_2 - 2R_1\): \(\begin{pmatrix} 1 & 2 & 3 \\ 0 & 0 & 0 \\ 1 & 2 & 5 \end{pmatrix}\)

\(R_3 \leftarrow R_3 - R_1\): \(\begin{pmatrix} 1 & 2 & 3 \\ 0 & 0 & 0 \\ 0 & 0 & 2 \end{pmatrix}\)

\(R_3 \leftrightarrow R_2\): \(\begin{pmatrix} 1 & 2 & 3 \\ 0 & 0 & 2 \\ 0 & 0 & 0 \end{pmatrix}\)

\(R_2 \leftarrow \frac{1}{2}R_2\): \(\begin{pmatrix} 1 & 2 & 3 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}\)

\(R_1 \leftarrow R_1 - 3R_2\): \(\begin{pmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}\)

Step 2: Identify pivot positions

In the row-reduced form \(\begin{pmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}\):

Pivot 1: Position (1,1) - leading 1 in row 1

Pivot 2: Position (2,3) - leading 1 in row 2

Total: 2 pivot positions

Step 3: Determine the rank

The rank equals the number of pivot positions.

Therefore: \(\text{rank}(A) = 2\)

Step 4: Verify using column space

Original columns: \(\vec{c_1} = \begin{pmatrix} 1 \\ 2 \\ 1 \end{pmatrix}\), \(\vec{c_2} = \begin{pmatrix} 2 \\ 4 \\ 2 \end{pmatrix}\), \(\vec{c_3} = \begin{pmatrix} 3 \\ 6 \\ 5 \end{pmatrix}\)

Notice: \(\vec{c_2} = 2\vec{c_1}\), so columns 1 and 2 are linearly dependent.

Also, \(\vec{c_3}\) is not a scalar multiple of \(\vec{c_1}\) (check: if \(\vec{c_3} = k\vec{c_1}\), then \(3 = k\), but then \(5 \neq k\cdot 1\)).

So \(\{\vec{c_1}, \vec{c_3}\}\) is a linearly independent set that spans the column space.

Dimension of column space = 2 = rank. ✓

\(\text{rank}(A) = 2\)
Final answer:

The rank of A is 2.

Applied rules:

• Row operations preserve rank: Elementary row operations don't change the rank

• Pivot counting: Number of pivots equals rank

• Linear independence: Count independent column vectors

2 Finding Nullity and Verifying Rank Theorem
Exercise 2
Find the nullity of the matrix from Exercise 1 and verify the Rank Theorem.
Definition:

Nullity of a Matrix: The nullity of an \(m \times n\) matrix A is the dimension of its null space, which equals the number of free variables in the homogeneous system \(A\vec{x} = \vec{0}\).

Row Reduced Form
\(\begin{pmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}\)
Free Variables
1 (x₂)
Rank Theorem
2 + 1 = 3 ✓
Step 1: Use the row-reduced form from Exercise 1

\(\begin{pmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}\)

Step 2: Identify pivot and free variables

From the row-reduced form: \(x_1 + 2x_2 = 0\) and \(x_3 = 0\)

Pivot variables (corresponding to pivot columns): \(x_1, x_3\)

Free variable: \(x_2\) (no pivot in column 2)

Step 3: Express solution in terms of free variables

From \(x_1 + 2x_2 = 0\): \(x_1 = -2x_2\)

From \(x_3 = 0\): \(x_3 = 0\)

Let \(x_2 = t\) where \(t\) is any real number.

Then: \(\begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \begin{pmatrix} -2t \\ t \\ 0 \end{pmatrix} = t\begin{pmatrix} -2 \\ 1 \\ 0 \end{pmatrix}\)

Step 4: Find the null space and its dimension

\(\text{null}(A) = \text{span}\left\{\begin{pmatrix} -2 \\ 1 \\ 0 \end{pmatrix}\right\}\)

Since the null space has a basis with 1 vector, \(\text{nullity}(A) = 1\)

Step 5: Verify the Rank Theorem

Matrix A is \(3 \times 3\), so \(n = 3\).

\(\text{rank}(A) + \text{nullity}(A) = 2 + 1 = 3 = n\). ✓

Step 6: Alternative calculation

Number of free variables = Total variables - Number of pivots = 3 - 2 = 1

Therefore, \(\text{nullity}(A) = 1\). ✓

\(\text{nullity}(A) = 1\), and \(\text{rank}(A) + \text{nullity}(A) = 2 + 1 = 3\)
Final answer:

The nullity of A is 1, and the Rank Theorem is verified: \(\text{rank}(A) + \text{nullity}(A) = 2 + 1 = 3\).

Applied rules:

• Free variable counting: Nullity equals number of free variables

• Rank-Nullity theorem: Fundamental relationship for verification

• Dimension calculation: Count vectors in null space basis

3 Full Rank Matrix
Exercise 3
Find the rank and nullity of \(B = \begin{pmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \end{pmatrix}\).
Definition:

Full Rank Matrix: A matrix is said to have full rank if its rank equals the minimum of its number of rows and columns. For an \(n \times n\) square matrix, full rank means \(\text{rank} = n\).

Matrix B
\(B = \begin{pmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \end{pmatrix}\)
Row Reduced
Already in echelon form
Rank and Nullity
3, 0
Step 1: Observe the matrix structure

\(B = \begin{pmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \end{pmatrix}\) is already in row echelon form.

Step 2: Identify pivot positions

In the matrix \(\begin{pmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \end{pmatrix}\):

Pivot 1: Position (1,1) - leading 1 in row 1

Pivot 2: Position (2,2) - leading 1 in row 2

Pivot 3: Position (3,3) - leading 1 in row 3

Total: 3 pivot positions

Step 3: Determine the rank

Since there are 3 pivot positions, \(\text{rank}(B) = 3\).

Step 4: Determine the nullity

Since every column has a pivot, there are no free variables.

Therefore: \(\text{nullity}(B) = 0\)

Step 5: Verify using Rank Theorem

Matrix B is \(3 \times 3\), so \(n = 3\).

\(\text{rank}(B) + \text{nullity}(B) = 3 + 0 = 3 = n\). ✓

Step 6: Verify by solving homogeneous system

Solve \(B\vec{x} = \vec{0}\): \(\begin{pmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \\ 0 & 0 & 1 \end{pmatrix}\begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}\)

This gives us: \(x_1 + 2x_3 = 0\), \(x_2 + 3x_3 = 0\), \(x_3 = 0\)

From the third equation: \(x_3 = 0\)

Substituting into the first: \(x_1 = 0\)

Substituting into the second: \(x_2 = 0\)

Only solution: \(\vec{x} = \vec{0}\), so null space is \(\{\vec{0}\}\) and nullity = 0. ✓

\(\text{rank}(B) = 3\), \(\text{nullity}(B) = 0\)
Final answer:

The rank of B is 3 and the nullity of B is 0. Matrix B has full rank.

Applied rules:

• Full rank matrix: All columns are linearly independent

• Trivial null space: Full rank ⟹ nullity = 0

• Triangular form: Easy to identify pivots in triangular matrices

Rank and Rank Theorem Properties
\(\text{rank}(A) + \text{nullity}(A) = n\)
Rank-Nullity Theorem
Rank Equality
\(\text{rank}(A) = \text{rank}(A^T)\)
Row rank equals column rank
Invertibility
\(A\) invertible ⟺ \(\text{rank}(A) = n\)
Full rank for square matrices
Bounds
\(\max(0, m+n-\min(m,n)) \leq \text{rank}(A) \leq \min(m,n)\)
Rank bounds for m×n matrix
Row Operations: Elementary row operations preserve the rank of a matrix
Column Operations: Elementary column operations also preserve rank
Linear Independence: Rank equals maximum number of linearly independent rows or columns
Solution: Exercises 4 to 5
4 Rank of Product
Exercise 4
If A is a \(3 \times 4\) matrix with \(\text{rank}(A) = 2\) and B is a \(4 \times 2\) matrix with \(\text{rank}(B) = 2\), what is the maximum possible rank of the product AB?
Definition:

Rank of Product: For matrices A and B where the product AB is defined, \(\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))\). This is because the column space of AB is contained in the column space of A.

Given Ranks
\(\text{rank}(A) = 2\), \(\text{rank}(B) = 2\)
Upper Bound
\(\text{rank}(AB) \leq \min(2,2) = 2\)
Maximum Possible
2
Step 1: Understand the constraint

For matrices A (size \(m \times n\)) and B (size \(n \times p\)), the product AB is defined and has size \(m \times p\).

Here: A is \(3 \times 4\) and B is \(4 \times 2\), so AB is \(3 \times 2\).

Step 2: Apply the rank inequality

The fundamental inequality for matrix ranks states:

\(\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))\)

This is because the column space of AB is contained in the column space of A.

Also, the row space of AB is contained in the row space of B.

Step 3: Calculate the maximum

Given: \(\text{rank}(A) = 2\) and \(\text{rank}(B) = 2\)

Therefore: \(\text{rank}(AB) \leq \min(2, 2) = 2\)

Step 4: Check if maximum is achievable

It's possible to construct examples where \(\text{rank}(AB) = 2\).

For example, if A and B are chosen such that the image of B is not in the null space of A, then the maximum rank can be achieved.

Step 5: Consider the dimensions

Since AB is \(3 \times 2\), the maximum possible rank is \(\min(3, 2) = 2\).

This agrees with our bound from the rank inequality.

Maximum possible rank of AB = 2
Final answer:

The maximum possible rank of AB is 2.

Applied rules:

• Rank inequality: \(\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))\)

• Dimension constraint: Rank ≤ min(rows, cols)

• Subspace containment: Col(AB) ⊆ Col(A) and Row(AB) ⊆ Row(B)

5 Rank Theorem Application
Exercise 5
Prove that for any \(m \times n\) matrix A, \(\text{rank}(A) \leq \min(m, n)\). When does equality occur?
Definition:

Rank Bounds: The rank of an \(m \times n\) matrix A is bounded by the number of rows and columns. The rank equals the dimension of the column space, which is a subspace of \(\mathbb{R}^m\), and also equals the dimension of the row space, which is a subspace of \(\mathbb{R}^n\).

Column Space
\(\text{rank}(A) = \dim(\text{col}(A)) \leq m\)
Row Space
\(\text{rank}(A) = \dim(\text{row}(A)) \leq n\)
Bound
\(\text{rank}(A) \leq \min(m,n)\)
Step 1: Consider the column space perspective

The rank of A equals the dimension of the column space of A.

The column space of A consists of all linear combinations of the columns of A.

Since each column of A has m entries, each column is in \(\mathbb{R}^m\).

Therefore, the column space is a subspace of \(\mathbb{R}^m\).

Hence: \(\text{rank}(A) = \dim(\text{col}(A)) \leq \dim(\mathbb{R}^m) = m\)

Step 2: Consider the row space perspective

The rank of A also equals the dimension of the row space of A.

The row space of A consists of all linear combinations of the rows of A.

Since A has n columns, each row has n entries, so each row is in \(\mathbb{R}^n\).

Therefore, the row space is a subspace of \(\mathbb{R}^n\).

Hence: \(\text{rank}(A) = \dim(\text{row}(A)) \leq \dim(\mathbb{R}^n) = n\)

Step 3: Combine the inequalities

From Steps 1 and 2: \(\text{rank}(A) \leq m\) and \(\text{rank}(A) \leq n\)

Therefore: \(\text{rank}(A) \leq \min(m, n)\)

Step 4: Determine when equality occurs

Equality \(\text{rank}(A) = \min(m, n)\) occurs when A has full rank.

Case 1: If \(m \geq n\), then \(\text{rank}(A) = n\) means A has full column rank (columns are linearly independent).

Case 2: If \(m \leq n\), then \(\text{rank}(A) = m\) means A has full row rank (rows are linearly independent).

Case 3: If \(m = n\), then \(\text{rank}(A) = n\) means A has full rank (both row and column rank are full), which is equivalent to A being invertible.

Step 5: Provide examples

Example where \(\text{rank}(A) = \min(m,n)\): For a \(3 \times 2\) matrix with 2 linearly independent columns, \(\text{rank} = 2 = \min(3,2)\).

Example where \(\text{rank}(A) < \min(m,n)\): For a \(3 \times 3\) matrix with rank 2, \(\text{rank} = 2 < 3 = \min(3,3)\).

\(\text{rank}(A) \leq \min(m,n)\) with equality when A has full rank.
Final answer:

For any \(m \times n\) matrix A, \(\text{rank}(A) \leq \min(m, n)\). Equality occurs when A has full rank: full column rank if \(m \geq n\), or full row rank if \(m \leq n\).

Applied rules:

• Subspace dimension: Dimension of subspace ≤ dimension of space

• Column-row rank equality: Row rank equals column rank

• Full rank characterization: When rank achieves maximum possible value

Rank and Rank Theorem: Advanced Concepts
\(\text{rank}(A) = \dim(\text{col}(A)) = \dim(\text{row}(A))\)
Rank Definition
Key definitions:

Rank: The dimension of the column space (or equivalently, the row space) of a matrix. It measures the maximum number of linearly independent rows or columns.

Nullity: The dimension of the null space of a matrix. It measures the number of degrees of freedom in the solution space of the homogeneous system.

Rank Theorem: For any \(m \times n\) matrix A: \(\text{rank}(A) + \text{nullity}(A) = n\).

Complete methodology:
  1. For rank: Row reduce to echelon form and count pivot positions
  2. For nullity: Count free variables or use rank-nullity theorem
  3. For verification: Use rank theorem as a consistency check
  4. For applications: Use rank to determine invertibility, consistency, etc.
Tip 1: The rank of a matrix never exceeds the minimum of its dimensions.
Tip 2: Use the rank theorem to check your calculations.
Tip 3: For square matrices, full rank implies invertibility.
Tip 4: Rank is preserved under elementary row and column operations.
Common errors: Forgetting to count pivots correctly, confusing rank with number of non-zero rows, misapplying the rank theorem.
Applications: Solving linear systems, matrix inversion, least squares, computer graphics, machine learning.
Properties to know by heart:

• \(\text{rank}(A) = \text{rank}(A^T)\) (row rank equals column rank)

• \(\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))\)

• \(A\) is invertible ⟺ \(\text{rank}(A) = n\) for \(n \times n\) matrix

• Homogeneous system \(A\vec{x} = \vec{0}\) has non-trivial solutions ⟺ \(\text{nullity}(A) > 0\)

• \(\text{rank}(A) \leq \min(m,n)\) for \(m \times n\) matrix A

Rank Visualization: Matrix Transformations
Exercise 6: Rank and Dimension Preservation
Visualize how a \(3 \times 3\) matrix with rank 2 transforms vectors in \(\mathbb{R}^3\). Show how the rank determines the dimension of the output space.

Geometric Interpretation: A matrix with rank r maps vectors to an r-dimensional subspace. For a rank-2 matrix, all outputs lie in a 2D plane within the codomain.

  • Input space: \(\mathbb{R}^3\) (3D)
  • Matrix rank: 2
  • Output space: 2D subspace of codomain
  • Null space: 1D (by rank theorem: 3-2=1)

Questions & Answers

Question: I'm confused about why row operations don't change the rank of a matrix. Shouldn't changing the rows change the row space?

Answer: This is a common misconception! Row operations do change individual rows, but they preserve the span of the row space:

Elementary row operations:

  • Row swapping: Doesn't change the set of vectors, just their order
  • Row scaling: Multiplies a row by a non-zero scalar, which doesn't change the span (since the scalar is invertible)
  • Row addition: Adds a multiple of one row to another, which adds a vector already in the span

For example, if you have rows \(\vec{r_1}, \vec{r_2}\), and you replace \(\vec{r_2}\) with \(\vec{r_2} + c\vec{r_1}\), the new row is still in the span of the original rows. So \(\text{span}\{\vec{r_1}, \vec{r_2}\} = \text{span}\{\vec{r_1}, \vec{r_2} + c\vec{r_1}\}\).

Since the row space remains unchanged, its dimension (the rank) remains the same.

Question: Why is the rank equal to both the row rank and the column rank? Isn't that surprising?

Answer: It is indeed surprising at first! This is one of the fundamental theorems of linear algebra. Here's the intuition:

Conceptual explanation:

  • The rank represents the "intrinsic dimension" of the linear transformation
  • Whether you count independent rows or independent columns, you're measuring the same underlying property
  • Row operations preserve the row space but also preserve the dimension of the column space

More formally, if you row reduce a matrix to echelon form, the number of pivots equals both:

  • The number of linearly independent rows (row rank)
  • The number of linearly independent columns (column rank)

This symmetry is fundamental to linear algebra and explains why we can simply speak of "the rank" of a matrix.

Question: What's the geometric interpretation of the Rank-Nullity Theorem?

Answer: The Rank-Nullity Theorem has a beautiful geometric interpretation:

Input space decomposition: Think of an \(m \times n\) matrix A as a linear transformation from \(\mathbb{R}^n\) to \(\mathbb{R}^m\).

The theorem says: \(\underbrace{\text{rank}}_{\text{output dimension}} + \underbrace{\text{nullity}}_{\text{collapsed dimension}} = \underbrace{n}_{\text{input dimension}}\)

Geometrically:

  • Rank: How many dimensions "survive" the transformation (dimension of output)
  • Nullity: How many dimensions "collapse" to zero (dimension of kernel)
  • Total: The original number of dimensions in the input space

It's like a conservation law for vector space dimensions: total input dimensions = preserved dimensions + collapsed dimensions.

Question: How is matrix rank used in machine learning and data science?

Answer: Matrix rank is fundamental in machine learning:

Principal Component Analysis (PCA):

  • Reduces dimension by keeping only high-rank components
  • The rank determines how many principal components are meaningful

Data Compression:

  • Low-rank approximations compress data efficiently
  • Singular Value Decomposition (SVD) uses rank concepts

Recommendation Systems:

  • User-item matrices often have low effective rank
  • Matrix completion techniques rely on rank assumptions

Model Complexity:

  • Rank measures the complexity of learned transformations
  • Regularization often penalizes high-rank solutions

Understanding rank helps in feature selection, overfitting prevention, and model interpretability.

Question: How does matrix rank relate to solving systems of linear equations in physics?

Answer: Matrix rank is crucial for understanding physical systems:

Consistency of systems:

  • \(A\vec{x} = \vec{b}\) has a solution ⟺ \(\text{rank}([A|\vec{b}]) = \text{rank}(A)\)
  • This determines if a physical system has a solution

Uniqueness of solutions:

  • If \(\text{rank}(A) = n\) (number of variables), solution is unique
  • If \(\text{rank}(A) < n\), solutions have \((n - \text{rank}(A))\) degrees of freedom

Physical interpretations:

  • In circuit theory: rank determines number of independent loops
  • In mechanics: rank determines number of independent constraints
  • In quantum mechanics: rank relates to the dimension of state space

Rank analysis helps determine solvability, uniqueness, and stability of physical models.