Solved Exercises on Linear Dependence and Independence

Master linear dependence, independence, and applications through these 5 detailed exercises.

Linear Dependence and Independence - Comprehensive Summary

Definitions

Linear Dependence: A set of vectors is linearly dependent if at least one vector can be written as a linear combination of the others

Linear Independence: A set of vectors is linearly independent if no vector can be written as a linear combination of the others

Trivial Solution: All coefficients equal zero in the linear combination equation

Core Rules

• Linear dependence: Non-trivial solution exists

• Linear independence: Only trivial solution exists

• Zero vector makes any set dependent

• Parallel vectors are dependent

Methods

1. Set up homogeneous system

2. Solve for coefficients

3. Check for non-trivial solutions

4. Apply determinant test (when applicable)

Key Tips

• More vectors than dimension = dependent

• Different directions = likely independent

• Check for scalar multiples

• Use row reduction for large sets

Solution: Exercises 1 to 3
1 Basic Linear Independence
Exercise 1
Determine if the vectors \(\vec{v_1} = \begin{pmatrix} 1 \\ 2 \end{pmatrix}\) and \(\vec{v_2} = \begin{pmatrix} 3 \\ 4 \end{pmatrix}\) are linearly independent.
Definition:

Linear Independence: Vectors \(\{\vec{v_1}, \vec{v_2}\}\) are linearly independent if the only solution to \(c_1\vec{v_1} + c_2\vec{v_2} = \vec{0}\) is \(c_1 = c_2 = 0\)

Linear Independence Method:
  1. Set up the homogeneous equation: \(c_1\vec{v_1} + c_2\vec{v_2} = \vec{0}\)
  2. Solve for the coefficients \(c_1, c_2\)
  3. If only the trivial solution exists, vectors are independent
Homogeneous System
\(\begin{cases} c_1 + 3c_2 = 0 \\ 2c_1 + 4c_2 = 0 \end{cases}\)
Solution
\(c_1 = c_2 = 0\)
Conclusion
Linearly Independent
Step 1: Set up the equation

We need to solve: \(c_1\begin{pmatrix} 1 \\ 2 \end{pmatrix} + c_2\begin{pmatrix} 3 \\ 4 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}\)

Step 2: Write the system of equations

\(\begin{cases} c_1 + 3c_2 = 0 \quad \text{(equation 1)} \\ 2c_1 + 4c_2 = 0 \quad \text{(equation 2)} \end{cases}\)

Step 3: Solve the system

From equation 1: \(c_1 = -3c_2\)

Substitute into equation 2: \(2(-3c_2) + 4c_2 = 0\)

\(-6c_2 + 4c_2 = 0\)

\(-2c_2 = 0\)

Therefore: \(c_2 = 0\), which gives \(c_1 = -3(0) = 0\)

Step 4: Conclusion

Since the only solution is \(c_1 = c_2 = 0\), the vectors are linearly independent.

The vectors \(\vec{v_1}\) and \(\vec{v_2}\) are linearly independent.
Final answer:

The vectors \(\vec{v_1}\) and \(\vec{v_2}\) are linearly independent.

Applied rules:

• Homogeneous system: Set linear combination equal to zero vector

• Trivial solution: Only zero coefficients indicate independence

• Unique solution: Non-zero coefficients would indicate dependence

2 Linear Dependence Example
Exercise 2
Determine if the vectors \(\vec{u_1} = \begin{pmatrix} 2 \\ 4 \end{pmatrix}\) and \(\vec{u_2} = \begin{pmatrix} 1 \\ 2 \end{pmatrix}\) are linearly dependent.
Definition:

Linear Dependence: Vectors \(\{\vec{u_1}, \vec{u_2}\}\) are linearly dependent if there exist scalars \(c_1, c_2\), not both zero, such that \(c_1\vec{u_1} + c_2\vec{u_2} = \vec{0}\)

Homogeneous System
\(\begin{cases} 2c_1 + c_2 = 0 \\ 4c_1 + 2c_2 = 0 \end{cases}\)
Solution
\(c_1 = -\frac{1}{2}c_2\)
Conclusion
Linearly Dependent
Step 1: Set up the equation

We need to solve: \(c_1\begin{pmatrix} 2 \\ 4 \end{pmatrix} + c_2\begin{pmatrix} 1 \\ 2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}\)

Step 2: Write the system of equations

\(\begin{cases} 2c_1 + c_2 = 0 \quad \text{(equation 1)} \\ 4c_1 + 2c_2 = 0 \quad \text{(equation 2)} \end{cases}\)

Step 3: Solve the system

From equation 1: \(c_2 = -2c_1\)

Substitute into equation 2: \(4c_1 + 2(-2c_1) = 0\)

\(4c_1 - 4c_1 = 0\)

\(0 = 0\)

This is always true, so there are infinitely many solutions.

One non-trivial solution: \(c_1 = 1, c_2 = -2\)

Verification: \(1\begin{pmatrix} 2 \\ 4 \end{pmatrix} + (-2)\begin{pmatrix} 1 \\ 2 \end{pmatrix} = \begin{pmatrix} 2 \\ 4 \end{pmatrix} + \begin{pmatrix} -2 \\ -4 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}\)

Step 4: Conclusion

Since there exists a non-trivial solution, the vectors are linearly dependent.

The vectors \(\vec{u_1}\) and \(\vec{u_2}\) are linearly dependent.
Final answer:

The vectors \(\vec{u_1}\) and \(\vec{u_2}\) are linearly dependent because \(\vec{u_1} = 2\vec{u_2}\).

Applied rules:

• Non-trivial solution: Existence indicates linear dependence

• Scalar relationship: One vector is a scalar multiple of another

• Verification: Always substitute back to confirm

3 Three Vector Independence
Exercise 3
Determine if the vectors \(\vec{w_1} = \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}\), \(\vec{w_2} = \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix}\), and \(\vec{w_3} = \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix}\) are linearly independent.
Definition:

Three Vector Independence: Vectors \(\{\vec{w_1}, \vec{w_2}, \vec{w_3}\}\) are linearly independent if the only solution to \(c_1\vec{w_1} + c_2\vec{w_2} + c_3\vec{w_3} = \vec{0}\) is \(c_1 = c_2 = c_3 = 0\)

Homogeneous System
\(\begin{cases} c_1 = 0 \\ c_2 = 0 \\ c_3 = 0 \end{cases}\)
Solution
\(c_1 = c_2 = c_3 = 0\)
Conclusion
Linearly Independent
Step 1: Set up the equation

We need to solve: \(c_1\begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix} + c_2\begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix} + c_3\begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}\)

Step 2: Write the system of equations

\(\begin{cases} c_1 + 0c_2 + 0c_3 = 0 \quad \text{(equation 1)} \\ 0c_1 + c_2 + 0c_3 = 0 \quad \text{(equation 2)} \\ 0c_1 + 0c_2 + c_3 = 0 \quad \text{(equation 3)} \end{cases}\)

Step 3: Solve the system

From equation 1: \(c_1 = 0\)

From equation 2: \(c_2 = 0\)

From equation 3: \(c_3 = 0\)

The only solution is \(c_1 = c_2 = c_3 = 0\)

Step 4: Conclusion

Since the only solution is the trivial solution, the vectors are linearly independent.

The vectors \(\vec{w_1}\), \(\vec{w_2}\), and \(\vec{w_3}\) are linearly independent.
Final answer:

The vectors \(\vec{w_1}\), \(\vec{w_2}\), and \(\vec{w_3}\) are linearly independent. They form the standard basis for \(\mathbb{R}^3\).

Applied rules:

• Standard basis: Unit vectors in different directions are independent

• Direct solution: Diagonal system has immediate solution

• Trivial solution: Only zero coefficients indicate independence

Linear Dependence and Independence Properties
\(\sum_{i=1}^{n} c_i\vec{v_i} = \vec{0} \Rightarrow \begin{cases} \text{Independent: } c_1 = ... = c_n = 0 \\ \text{Dependent: } \exists j, c_j \neq 0 \end{cases}\)
Independence Definition
Zero Vector
\(\text{If } \vec{0} \in S \text{, then } S \text{ is dependent}\)
Zero vector always makes a set dependent
Scalar Multiple
\(\text{If } \vec{v_2} = k\vec{v_1} \text{, then } \{\vec{v_1}, \vec{v_2}\} \text{ is dependent}\)
Parallel vectors are dependent
Maximum Independence
\text{Max } n \text{ independent vectors in } \mathbb{R}^n\)
Dimension limits independence
Geometric Interpretation: Independent vectors point in different directions and span a larger space than dependent vectors
Matrix Test: Vectors are independent if and only if the matrix formed by them has full rank
Determinant Test: For n vectors in \(\mathbb{R}^n\), they are independent if and only if the determinant of the matrix is non-zero
Solution: Exercises 4 to 5
4 Four Vector Dependence
Exercise 4
Determine if the vectors \(\vec{v_1} = \begin{pmatrix} 1 \\ 2 \\ 1 \\ 0 \end{pmatrix}\), \(\vec{v_2} = \begin{pmatrix} 2 \\ 1 \\ 0 \\ 1 \end{pmatrix}\), \(\vec{v_3} = \begin{pmatrix} 3 \\ 3 \\ 1 \\ 1 \end{pmatrix}\), and \(\vec{v_4} = \begin{pmatrix} 0 \\ 1 \\ 1 \\ 1 \end{pmatrix}\) are linearly independent.
Definition:

Four Vector Independence: Check if the only solution to \(c_1\vec{v_1} + c_2\vec{v_2} + c_3\vec{v_3} + c_4\vec{v_4} = \vec{0}\) is \(c_1 = c_2 = c_3 = c_4 = 0\)

Homogeneous System
\(\begin{cases} c_1 + 2c_2 + 3c_3 = 0 \\ 2c_1 + c_2 + 3c_3 + c_4 = 0 \\ c_1 + c_3 + c_4 = 0 \\ c_2 + c_3 + c_4 = 0 \end{cases}\)
Solution
Non-trivial solution exists
Conclusion
Linearly Dependent
Step 1: Set up the equation

We need to solve: \(c_1\begin{pmatrix} 1 \\ 2 \\ 1 \\ 0 \end{pmatrix} + c_2\begin{pmatrix} 2 \\ 1 \\ 0 \\ 1 \end{pmatrix} + c_3\begin{pmatrix} 3 \\ 3 \\ 1 \\ 1 \end{pmatrix} + c_4\begin{pmatrix} 0 \\ 1 \\ 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \end{pmatrix}\)

Step 2: Write the system of equations

\(\begin{cases} c_1 + 2c_2 + 3c_3 + 0c_4 = 0 \quad \text{(equation 1)} \\ 2c_1 + c_2 + 3c_3 + c_4 = 0 \quad \text{(equation 2)} \\ c_1 + 0c_2 + c_3 + c_4 = 0 \quad \text{(equation 3)} \\ 0c_1 + c_2 + c_3 + c_4 = 0 \quad \text{(equation 4)} \end{cases}\)

Step 3: Solve the system

From equation 3: \(c_1 = -c_3 - c_4\)

From equation 4: \(c_2 = -c_3 - c_4\)

So: \(c_1 = c_2\)

Substitute into equation 1: \((-c_3 - c_4) + 2(-c_3 - c_4) + 3c_3 = 0\)

\(-c_3 - c_4 - 2c_3 - 2c_4 + 3c_3 = 0\)

\(-3c_4 = 0\)

Therefore: \(c_4 = 0\)

This gives: \(c_1 = -c_3\) and \(c_2 = -c_3\)

Substitute into equation 2: \(2(-c_3) + (-c_3) + 3c_3 + 0 = 0\)

\(-2c_3 - c_3 + 3c_3 = 0\)

\(0 = 0\)

This is always true, so we can choose \(c_3\) freely.

Let \(c_3 = 1\), then: \(c_1 = -1\), \(c_2 = -1\), \(c_3 = 1\), \(c_4 = 0\)

Step 4: Verify the solution

\((-1)\begin{pmatrix} 1 \\ 2 \\ 1 \\ 0 \end{pmatrix} + (-1)\begin{pmatrix} 2 \\ 1 \\ 0 \\ 1 \end{pmatrix} + (1)\begin{pmatrix} 3 \\ 3 \\ 1 \\ 1 \end{pmatrix} + (0)\begin{pmatrix} 0 \\ 1 \\ 1 \\ 1 \end{pmatrix}\)

\(= \begin{pmatrix} -1 \\ -2 \\ -1 \\ 0 \end{pmatrix} + \begin{pmatrix} -2 \\ -1 \\ 0 \\ -1 \end{pmatrix} + \begin{pmatrix} 3 \\ 3 \\ 1 \\ 1 \end{pmatrix} + \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \end{pmatrix}\)

The vectors are linearly dependent with non-trivial solution \(c_1 = -1, c_2 = -1, c_3 = 1, c_4 = 0\).
Final answer:

The vectors are linearly dependent because we found a non-trivial solution to the homogeneous equation.

Applied rules:

• More vectors than dimension: 4 vectors in \(\mathbb{R}^4\) may still be dependent

• Free variables: Indicate infinite solutions and dependence

• Verification: Always substitute back to confirm

5 Independence with Parameters
Exercise 5
For what values of the parameter \(k\) are the vectors \(\vec{a_1} = \begin{pmatrix} 1 \\ 2 \end{pmatrix}\) and \(\vec{a_2} = \begin{pmatrix} k \\ 3 \end{pmatrix}\) linearly independent?
Definition:

Parameterized Independence: Determine parameter values for which the only solution to \(c_1\vec{a_1} + c_2\vec{a_2} = \vec{0}\) is \(c_1 = c_2 = 0\)

Homogeneous System
\(\begin{cases} c_1 + kc_2 = 0 \\ 2c_1 + 3c_2 = 0 \end{cases}\)
Condition
\(k \neq \frac{3}{2}\)
Conclusion
Independent when \(k \neq \frac{3}{2}\)
Step 1: Set up the equation

We need to solve: \(c_1\begin{pmatrix} 1 \\ 2 \end{pmatrix} + c_2\begin{pmatrix} k \\ 3 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}\)

Step 2: Write the system of equations

\(\begin{cases} c_1 + kc_2 = 0 \quad \text{(equation 1)} \\ 2c_1 + 3c_2 = 0 \quad \text{(equation 2)} \end{cases}\)

Step 3: Solve the system

From equation 2: \(c_1 = -\frac{3}{2}c_2\)

Substitute into equation 1: \(-\frac{3}{2}c_2 + kc_2 = 0\)

\(c_2(-\frac{3}{2} + k) = 0\)

For independence, we need the only solution to be \(c_2 = 0\), which means \(c_1 = 0\).

This occurs when \(-\frac{3}{2} + k \neq 0\), i.e., when \(k \neq \frac{3}{2}\).

If \(k = \frac{3}{2}\), then any value of \(c_2\) works, giving infinitely many solutions.

Step 4: Conclusion

The vectors are linearly independent when \(k \neq \frac{3}{2}\) and linearly dependent when \(k = \frac{3}{2}\).

The vectors are linearly independent for all \(k \neq \frac{3}{2}\).
Final answer:

The vectors \(\vec{a_1}\) and \(\vec{a_2}\) are linearly independent if and only if \(k \neq \frac{3}{2}\).

Applied rules:

• Parameter conditions: Determine critical values where independence changes

• Division by zero: Be careful when dividing by expressions involving parameters

• Critical values: Values where the system behavior changes

Linear Dependence and Independence: Advanced Concepts
\(c_1\vec{v_1} + c_2\vec{v_2} + ... + c_n\vec{v_n} = \vec{0} \Leftrightarrow c_1 = c_2 = ... = c_n = 0 \text{ (independent)}\)
Independence Criterion
Key definitions:

Linear Independence: A set of vectors where no vector can be expressed as a linear combination of the others

Linear Dependence: A set of vectors where at least one vector can be expressed as a linear combination of the others

Trivial Solution: The solution where all coefficients are zero in a homogeneous linear combination

Complete methodology:
  1. Set up homogeneous system: \(c_1\vec{v_1} + ... + c_n\vec{v_n} = \vec{0}\)
  2. Solve for coefficients: Use substitution, elimination, or matrix methods
  3. Check for non-trivial solutions: If any coefficient can be non-zero, vectors are dependent
  4. Alternative tests: Use determinant or rank tests when applicable
Tip 1: Two vectors are dependent if one is a scalar multiple of the other.
Tip 2: Any set containing the zero vector is linearly dependent.
Tip 3: If you have more vectors than the dimension of the space, they must be dependent.
Tip 4: Use row reduction for larger systems to determine independence efficiently.
Common errors: Misidentifying the homogeneous system, forgetting to verify solutions, not considering all possible solutions.
Applications: Basis determination, dimension calculation, coordinate systems, computer graphics, machine learning.
Properties to know by heart:

• A single non-zero vector is always linearly independent

• Two vectors are dependent if and only if one is a scalar multiple of the other

• A set of vectors is dependent if any subset is dependent

• In \(\mathbb{R}^n\), at most \(n\) vectors can be linearly independent

• Linearly independent vectors form a basis for their span

Linear Dependence Visualization: Geometric Interpretation
Exercise 6: Geometric Independence
Visualize the difference between linearly independent and dependent vectors in \(\mathbb{R}^2\). Show that independent vectors span the entire plane while dependent vectors span only a line.

Geometric Interpretation: Linearly independent vectors in \(\mathbb{R}^2\) span the entire plane, while dependent vectors span only a line through the origin.

  • Independent: \(\vec{v_1} = \begin{pmatrix} 1 \\ 0 \end{pmatrix}\) and \(\vec{v_2} = \begin{pmatrix} 0 \\ 1 \end{pmatrix}\) span \(\mathbb{R}^2\)
  • Dependent: \(\vec{u_1} = \begin{pmatrix} 2 \\ 4 \end{pmatrix}\) and \(\vec{u_2} = \begin{pmatrix} 1 \\ 2 \end{pmatrix}\) span a line

Questions & Answers

Question: I'm confused about why we set the linear combination equal to zero when checking for independence. Why not equal to some other vector?

Answer: Great question! The reason we use the zero vector is fundamental to the definition of linear independence:

  • Independence: We want to know if any vector can be written as a combination of the others. If \(\vec{v_k} = c_1\vec{v_1} + ... + c_{k-1}\vec{v_{k-1}} + c_{k+1}\vec{v_{k+1}} + ... + c_n\vec{v_n}\), then rearranging gives: \(c_1\vec{v_1} + ... + (-1)\vec{v_k} + ... + c_n\vec{v_n} = \vec{0}\)
  • Homogeneous system: Setting equal to zero isolates the coefficients and reveals whether there's a non-trivial relationship among the vectors
  • Trivial vs. non-trivial: The zero vector allows us to distinguish between the "trivial" solution (all coefficients zero) and "non-trivial" solutions (some coefficients non-zero)

If we set the combination equal to some other vector \(\vec{b}\), we'd be solving a different problem about span membership, not independence.

Question: How does linear independence relate to the concept of a basis?

Answer: Linear independence is one of the two key requirements for a basis:

  1. Linear independence: The vectors in a basis must be linearly independent, meaning no vector can be written as a linear combination of the others
  2. Spanning property: The vectors in a basis must span the entire vector space, meaning every vector in the space can be written as a linear combination of the basis vectors

So a basis is a linearly independent set that also spans the space. For example, in \(\mathbb{R}^3\):

  • The standard basis \(\left\{\begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}, \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix}, \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix}\right\}\) is linearly independent and spans \(\mathbb{R}^3\)
  • Any set of 3 linearly independent vectors in \(\mathbb{R}^3\) automatically spans the space and forms a basis

Question: What's the fastest way to check linear independence for a large set of vectors?

Answer: For efficiency with large sets, use these approaches:

  1. Immediate checks: Look for obvious dependencies (zero vectors, scalar multiples, repeated vectors)
  2. Count comparison: If you have more vectors than the dimension of the space, they're automatically dependent
  3. Row reduction: Form a matrix with the vectors as columns and reduce to row echelon form. Count pivot positions.
  4. Determinant test: For \(n\) vectors in \(\mathbb{R}^n\), compute the determinant of the matrix they form. Non-zero = independent.

The row reduction method is most reliable: if the number of pivot positions equals the number of vectors, they're independent. If fewer pivots than vectors, they're dependent. This is computationally efficient and works for any size system.

Question: How is linear independence used in computer graphics?

Answer: Linear independence is crucial in computer graphics for:

  • Coordinate systems: Ensuring that basis vectors for local coordinate systems are independent to avoid degenerate transformations
  • Surface normals: Checking independence of edge vectors to properly compute surface normals
  • Texture mapping: Ensuring texture coordinate vectors are independent for proper UV mapping
  • Animation: Maintaining independence in blend shape vectors to prevent unwanted deformations

For example, when defining a local coordinate system for an object, we need three linearly independent vectors (usually mutually perpendicular) to ensure the transformation matrix is invertible and doesn't collapse the space.

Question: How does linear independence affect machine learning models?

Answer: Linear independence has significant impact on ML models:

  1. Feature selection: Linearly dependent features provide redundant information and can cause multicollinearity
  2. Model stability: Dependent features can make model training numerically unstable
  3. Interpretability: Independent features allow clearer interpretation of model coefficients
  4. Dimensionality reduction: Techniques like PCA find linearly independent directions of maximum variance

In regression models, if features are linearly dependent, the normal equations may not have a unique solution. Regularization techniques are often needed to handle near-dependencies in real-world data.