Solved Exercises on Linear Combinations and Span

Master linear combinations, span, linear independence, and applications through these 5 detailed exercises.

Linear Combinations and Span - Comprehensive Summary

Definitions

Linear Combination: A sum of vectors, each multiplied by a scalar: \(c_1\vec{v_1} + c_2\vec{v_2} + ... + c_n\vec{v_n}\)

Span: The set of all possible linear combinations of a given set of vectors

Linear Independence: Vectors are independent if no vector can be written as a linear combination of others

Core Rules

• Scalars determine weights in linear combinations

• Span includes all possible linear combinations

• Independent vectors span larger subspaces

• Dependent vectors span smaller subspaces

Methods

1. Set up linear system equation

2. Solve for coefficients

3. Check for consistency

4. Determine span membership

Key Tips

• Check if vectors span entire space

• Verify linear independence

• Use matrix methods for efficiency

• Visualize in 2D/3D when possible

Solution: Exercises 1 to 3
1 Basic Linear Combination
Exercise 1
Let \(\vec{v_1} = \begin{pmatrix} 1 \\ 2 \end{pmatrix}\) and \(\vec{v_2} = \begin{pmatrix} 3 \\ 1 \end{pmatrix}\). Find the linear combination \(2\vec{v_1} - 3\vec{v_2}\).
Definition:

Linear Combination: A sum of vectors, each multiplied by a scalar: \(c_1\vec{v_1} + c_2\vec{v_2} + ... + c_n\vec{v_n}\)

Linear Combination Method:
  1. Multiply each vector by its corresponding scalar
  2. Perform vector addition/subtraction component-wise
  3. Simplify the result
Given Vectors
\(\vec{v_1} = \begin{pmatrix} 1 \\ 2 \end{pmatrix}, \vec{v_2} = \begin{pmatrix} 3 \\ 1 \end{pmatrix}\)
Scalars
\(c_1 = 2, c_2 = -3\)
Result
\(\begin{pmatrix} -7 \\ 1 \end{pmatrix}\)
Step 1: Calculate scalar multiplications

\(2\vec{v_1} = 2 \cdot \begin{pmatrix} 1 \\ 2 \end{pmatrix} = \begin{pmatrix} 2 \\ 4 \end{pmatrix}\)

\(-3\vec{v_2} = -3 \cdot \begin{pmatrix} 3 \\ 1 \end{pmatrix} = \begin{pmatrix} -9 \\ -3 \end{pmatrix}\)

Step 2: Add the results

\(2\vec{v_1} - 3\vec{v_2} = \begin{pmatrix} 2 \\ 4 \end{pmatrix} + \begin{pmatrix} -9 \\ -3 \end{pmatrix} = \begin{pmatrix} 2-9 \\ 4-3 \end{pmatrix}\)

Step 3: Simplify

\(2\vec{v_1} - 3\vec{v_2} = \begin{pmatrix} -7 \\ 1 \end{pmatrix}\)

\(2\vec{v_1} - 3\vec{v_2} = \begin{pmatrix} -7 \\ 1 \end{pmatrix}\)
Final answer:

\(2\vec{v_1} - 3\vec{v_2} = \begin{pmatrix} -7 \\ 1 \end{pmatrix}\)

Applied rules:

• Scalar multiplication: Multiply each component by the scalar

• Component-wise addition: Add corresponding components

• Sign handling: Pay attention to negative scalars

2 Span Membership
Exercise 2
Determine if \(\vec{b} = \begin{pmatrix} 7 \\ 5 \end{pmatrix}\) is in the span of \(\{\vec{v_1}, \vec{v_2}\}\) where \(\vec{v_1} = \begin{pmatrix} 1 \\ 2 \end{pmatrix}\) and \(\vec{v_2} = \begin{pmatrix} 3 \\ 1 \end{pmatrix}\).
Definition:

Span Membership: A vector \(\vec{b}\) is in span\(\{\vec{v_1}, \vec{v_2}\}\) if there exist scalars \(c_1, c_2\) such that \(\vec{b} = c_1\vec{v_1} + c_2\vec{v_2}\)

System
\(\begin{cases} c_1 + 3c_2 = 7 \\ 2c_1 + c_2 = 5 \end{cases}\)
Solution
\(c_1 = 2, c_2 = \frac{5}{3}\)
Membership
\(\vec{b} \in \text{span}\{\vec{v_1}, \vec{v_2}\}\)
Step 1: Set up the equation

We need to find \(c_1, c_2\) such that: \(\begin{pmatrix} 7 \\ 5 \end{pmatrix} = c_1\begin{pmatrix} 1 \\ 2 \end{pmatrix} + c_2\begin{pmatrix} 3 \\ 1 \end{pmatrix}\)

Step 2: Write the system of equations

\(\begin{cases} c_1 + 3c_2 = 7 \quad \text{(equation 1)} \\ 2c_1 + c_2 = 5 \quad \text{(equation 2)} \end{cases}\)

Step 3: Solve the system

From equation 2: \(c_2 = 5 - 2c_1\)

Substitute into equation 1: \(c_1 + 3(5 - 2c_1) = 7\)

\(c_1 + 15 - 6c_1 = 7\)

\(-5c_1 = -8\)

\(c_1 = \frac{8}{5}\)

Then: \(c_2 = 5 - 2(\frac{8}{5}) = 5 - \frac{16}{5} = \frac{25-16}{5} = \frac{9}{5}\)

Step 4: Verify the solution

\(\frac{8}{5}\begin{pmatrix} 1 \\ 2 \end{pmatrix} + \frac{9}{5}\begin{pmatrix} 3 \\ 1 \end{pmatrix} = \begin{pmatrix} \frac{8}{5} \\ \frac{16}{5} \end{pmatrix} + \begin{pmatrix} \frac{27}{5} \\ \frac{9}{5} \end{pmatrix} = \begin{pmatrix} \frac{35}{5} \\ \frac{25}{5} \end{pmatrix} = \begin{pmatrix} 7 \\ 5 \end{pmatrix}\)

\(\vec{b} = \frac{8}{5}\vec{v_1} + \frac{9}{5}\vec{v_2}\), so \(\vec{b} \in \text{span}\{\vec{v_1}, \vec{v_2}\}\)
Final answer:

Yes, \(\vec{b}\) is in the span of \(\{\vec{v_1}, \vec{v_2}\}\) with coefficients \(c_1 = \frac{8}{5}\) and \(c_2 = \frac{9}{5}\)

Applied rules:

• Span definition: Express target vector as linear combination

• System solving: Solve for coefficients using substitution/elimination

• Verification: Substitute back to confirm the solution

3 3D Linear Combination
Exercise 3
Find the linear combination \(3\vec{u} - 2\vec{v} + \vec{w}\) where \(\vec{u} = \begin{pmatrix} 1 \\ 0 \\ -1 \end{pmatrix}\), \(\vec{v} = \begin{pmatrix} 2 \\ 1 \\ 3 \end{pmatrix}\), and \(\vec{w} = \begin{pmatrix} 0 \\ -1 \\ 2 \end{pmatrix}\).
Definition:

Multi-vector Linear Combination: Extends to any number of vectors: \(\sum_{i=1}^{n} c_i\vec{v_i}\)

Given Vectors
\(\vec{u} = \begin{pmatrix} 1 \\ 0 \\ -1 \end{pmatrix}, \vec{v} = \begin{pmatrix} 2 \\ 1 \\ 3 \end{pmatrix}, \vec{w} = \begin{pmatrix} 0 \\ -1 \\ 2 \end{pmatrix}\)
Scalar Mult
\(3\vec{u} = \begin{pmatrix} 3 \\ 0 \\ -3 \end{pmatrix}, -2\vec{v} = \begin{pmatrix} -4 \\ -2 \\ -6 \end{pmatrix}, \vec{w} = \begin{pmatrix} 0 \\ -1 \\ 2 \end{pmatrix}\)
Result
\(\begin{pmatrix} -1 \\ -3 \\ -7 \end{pmatrix}\)
Step 1: Calculate scalar multiplications

\(3\vec{u} = 3 \cdot \begin{pmatrix} 1 \\ 0 \\ -1 \end{pmatrix} = \begin{pmatrix} 3 \\ 0 \\ -3 \end{pmatrix}\)

\(-2\vec{v} = -2 \cdot \begin{pmatrix} 2 \\ 1 \\ 3 \end{pmatrix} = \begin{pmatrix} -4 \\ -2 \\ -6 \end{pmatrix}\)

\(\vec{w} = \begin{pmatrix} 0 \\ -1 \\ 2 \end{pmatrix}\)

Step 2: Add all vectors component-wise

\(3\vec{u} - 2\vec{v} + \vec{w} = \begin{pmatrix} 3 \\ 0 \\ -3 \end{pmatrix} + \begin{pmatrix} -4 \\ -2 \\ -6 \end{pmatrix} + \begin{pmatrix} 0 \\ -1 \\ 2 \end{pmatrix}\)

\(= \begin{pmatrix} 3-4+0 \\ 0-2-1 \\ -3-6+2 \end{pmatrix} = \begin{pmatrix} -1 \\ -3 \\ -7 \end{pmatrix}\)

Step 3: Verify each component

First: \(3-4+0 = -1\)

Second: \(0-2-1 = -3\)

Third: \(-3-6+2 = -7\)

\(3\vec{u} - 2\vec{v} + \vec{w} = \begin{pmatrix} -1 \\ -3 \\ -7 \end{pmatrix}\)
Final answer:

\(3\vec{u} - 2\vec{v} + \vec{w} = \begin{pmatrix} -1 \\ -3 \\ -7 \end{pmatrix}\)

Applied rules:

• Extension to higher dimensions: Rules apply to n-dimensional vectors

• Sequential operations: Perform scalar multiplication first, then addition

• Component-wise operations: Apply operations to each component separately

Linear Combinations and Span Properties
\(\text{span}\{\vec{v_1}, \vec{v_2}, ..., \vec{v_n}\} = \left\{ \sum_{i=1}^{n} c_i\vec{v_i} \mid c_i \in \mathbb{R} \right\}\)
Span Definition
Zero Vector
\(\vec{0} \in \text{span}\{\vec{v_1}, ..., \vec{v_n}\}\)
The zero vector is always in the span
Closure
\(\vec{u}, \vec{v} \in S \Rightarrow \vec{u}+\vec{v} \in S\)
Span is closed under addition
Scalar Closure
\(\vec{u} \in S, c \in \mathbb{R} \Rightarrow c\vec{u} \in S\)
Span is closed under scalar multiplication
Linear Independence: Vectors are linearly independent if the only solution to \(c_1\vec{v_1} + ... + c_n\vec{v_n} = \vec{0}\) is \(c_1 = ... = c_n = 0\)
Spanning Set: A set of vectors spans a space if every vector in the space can be written as a linear combination of those vectors
Basis: A linearly independent spanning set is called a basis for the space
Solution: Exercises 4 to 5
4 Span of Three Vectors
Exercise 4
Determine if \(\vec{b} = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}\) is in the span of \(\{\vec{v_1}, \vec{v_2}, \vec{v_3}\}\) where \(\vec{v_1} = \begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix}\), \(\vec{v_2} = \begin{pmatrix} 0 \\ 1 \\ 1 \end{pmatrix}\), \(\vec{v_3} = \begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix}\).
Definition:

3D Span Membership: Check if there exist scalars \(c_1, c_2, c_3\) such that \(\vec{b} = c_1\vec{v_1} + c_2\vec{v_2} + c_3\vec{v_3}\)

System
\(\begin{cases} c_1 + c_3 = 1 \\ c_2 + c_3 = 2 \\ c_1 + c_2 = 3 \end{cases}\)
Solution
\(c_1 = 1, c_2 = 2, c_3 = 0\)
Membership
\(\vec{b} \in \text{span}\{\vec{v_1}, \vec{v_2}, \vec{v_3}\}\)
Step 1: Set up the equation

We need to find \(c_1, c_2, c_3\) such that: \(\begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix} = c_1\begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix} + c_2\begin{pmatrix} 0 \\ 1 \\ 1 \end{pmatrix} + c_3\begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix}\)

Step 2: Write the system of equations

\(\begin{cases} c_1 + 0c_2 + c_3 = 1 \quad \text{(equation 1)} \\ 0c_1 + c_2 + c_3 = 2 \quad \text{(equation 2)} \\ c_1 + c_2 + 0c_3 = 3 \quad \text{(equation 3)} \end{cases}\)

Step 3: Solve the system

From equation 1: \(c_1 = 1 - c_3\)

From equation 2: \(c_2 = 2 - c_3\)

Substitute into equation 3: \((1 - c_3) + (2 - c_3) = 3\)

\(3 - 2c_3 = 3\)

\(-2c_3 = 0\)

\(c_3 = 0\)

Therefore: \(c_1 = 1 - 0 = 1\) and \(c_2 = 2 - 0 = 2\)

Step 4: Verify the solution

\(1\begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix} + 2\begin{pmatrix} 0 \\ 1 \\ 1 \end{pmatrix} + 0\begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix} + \begin{pmatrix} 0 \\ 2 \\ 2 \end{pmatrix} + \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix} = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}\)

\(\vec{b} = 1\vec{v_1} + 2\vec{v_2} + 0\vec{v_3}\), so \(\vec{b} \in \text{span}\{\vec{v_1}, \vec{v_2}, \vec{v_3}\}\)
Final answer:

Yes, \(\vec{b}\) is in the span of \(\{\vec{v_1}, \vec{v_2}, \vec{v_3}\}\) with coefficients \(c_1 = 1\), \(c_2 = 2\), \(c_3 = 0\)

Applied rules:

• System extension: More vectors mean more equations

• Consistent solution: A unique solution means the vector is in the span

• Verification: Always substitute back to confirm

5 Linear Independence Check
Exercise 5
Determine if the vectors \(\vec{v_1} = \begin{pmatrix} 1 \\ 2 \\ 1 \end{pmatrix}\), \(\vec{v_2} = \begin{pmatrix} 2 \\ 1 \\ 3 \end{pmatrix}\), \(\vec{v_3} = \begin{pmatrix} 3 \\ 4 \\ 5 \end{pmatrix}\) are linearly independent.
Definition:

Linear Independence: Vectors are linearly independent if the only solution to \(c_1\vec{v_1} + c_2\vec{v_2} + c_3\vec{v_3} = \vec{0}\) is \(c_1 = c_2 = c_3 = 0\)

Homogeneous System
\(\begin{cases} c_1 + 2c_2 + 3c_3 = 0 \\ 2c_1 + c_2 + 4c_3 = 0 \\ c_1 + 3c_2 + 5c_3 = 0 \end{cases}\)
Solution
\(c_1 = c_2 = c_3 = 0\)
Independence
Linearly Independent
Step 1: Set up the homogeneous system

We need to solve: \(c_1\begin{pmatrix} 1 \\ 2 \\ 1 \end{pmatrix} + c_2\begin{pmatrix} 2 \\ 1 \\ 3 \end{pmatrix} + c_3\begin{pmatrix} 3 \\ 4 \\ 5 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}\)

Step 2: Write the system of equations

\(\begin{cases} c_1 + 2c_2 + 3c_3 = 0 \quad \text{(equation 1)} \\ 2c_1 + c_2 + 4c_3 = 0 \quad \text{(equation 2)} \\ c_1 + 3c_2 + 5c_3 = 0 \quad \text{(equation 3)} \end{cases}\)

Step 3: Solve the system

From equation 1: \(c_1 = -2c_2 - 3c_3\)

Substitute into equation 2: \(2(-2c_2 - 3c_3) + c_2 + 4c_3 = 0\)

\(-4c_2 - 6c_3 + c_2 + 4c_3 = 0\)

\(-3c_2 - 2c_3 = 0\)

So: \(c_2 = -\frac{2}{3}c_3\)

Substitute into equation 3: \((-2(-\frac{2}{3}c_3) - 3c_3) + 3(-\frac{2}{3}c_3) + 5c_3 = 0\)

\((\frac{4}{3}c_3 - 3c_3) - 2c_3 + 5c_3 = 0\)

\(\frac{4}{3}c_3 - 3c_3 - 2c_3 + 5c_3 = 0\)

\(\frac{4}{3}c_3 = 0\)

Therefore: \(c_3 = 0\), which gives \(c_2 = 0\) and \(c_1 = 0\)

Step 4: Conclusion

Since the only solution is \(c_1 = c_2 = c_3 = 0\), the vectors are linearly independent.

The vectors are linearly independent since only the trivial solution exists.
Final answer:

The vectors \(\vec{v_1}\), \(\vec{v_2}\), and \(\vec{v_3}\) are linearly independent.

Applied rules:

• Homogeneous system: Set linear combination equal to zero vector

• Trivial solution: Only zero coefficients indicate independence

• Non-trivial solution: Non-zero coefficients indicate dependence

Linear Combinations and Span: Advanced Concepts
\(\vec{b} = c_1\vec{v_1} + c_2\vec{v_2} + ... + c_n\vec{v_n}\)
Linear Combination
Key definitions:

Linear Combination: A weighted sum of vectors using scalar coefficients

Span: The set of all possible linear combinations of a given set of vectors

Linear Independence: A set of vectors where no vector can be expressed as a linear combination of others

Complete methodology:
  1. For span membership: Set up system of equations with unknown coefficients
  2. For linear independence: Solve homogeneous system for zero vector
  3. For linear combinations: Apply scalar multiplication followed by vector addition
  4. Verify solutions: Always substitute back to confirm
Tip 1: In 2D, two non-parallel vectors span the entire plane if they're linearly independent.
Tip 2: If one vector is a scalar multiple of another, they are linearly dependent.
Tip 3: The span of a single vector is a line through the origin in the direction of that vector.
Tip 4: Use row reduction for larger systems to determine linear independence efficiently.
Common errors: Forgetting to verify solutions, misidentifying linear independence, incorrectly setting up systems of equations.
Applications: Computer graphics, signal processing, machine learning, physics, economics.
Properties to know by heart:

• Every vector space contains the zero vector

• Span is always a subspace of the original vector space

• Linearly independent sets have unique representations

• The maximum number of linearly independent vectors in \(\mathbb{R}^n\) is \(n\)

• Basis vectors are linearly independent and span the entire space

Linear Combinations Visualization: Geometric Interpretation
Exercise 6: Span Visualization
Visualize the span of vectors \(\vec{v_1} = \begin{pmatrix} 1 \\ 0 \end{pmatrix}\) and \(\vec{v_2} = \begin{pmatrix} 0 \\ 1 \end{pmatrix}\). Show that their span covers the entire \(\mathbb{R}^2\) plane.

Geometric Interpretation: The span of two linearly independent vectors in \(\mathbb{R}^2\) creates the entire plane, allowing any vector in \(\mathbb{R}^2\) to be expressed as a linear combination of these vectors.

  • \(\vec{v_1} = \begin{pmatrix} 1 \\ 0 \end{pmatrix}\) (unit vector along x-axis)
  • \(\vec{v_2} = \begin{pmatrix} 0 \\ 1 \end{pmatrix}\) (unit vector along y-axis)
  • Any vector \(\vec{w} = \begin{pmatrix} a \\ b \end{pmatrix} = a\vec{v_1} + b\vec{v_2}\)

Questions & Answers

Question: I'm having trouble understanding the difference between linear combinations and span. Can you explain?

Answer: Great question! Think of it this way:

  • Linear combination: A specific instance of adding scaled vectors. For example, \(3\vec{v_1} + 2\vec{v_2}\) is one particular linear combination.
  • Span: The collection of ALL possible linear combinations. It's the entire set of vectors you can create using the given vectors.

If you have vectors \(\vec{v_1}\) and \(\vec{v_2}\), then:

  • One linear combination might be: \(2\vec{v_1} + 5\vec{v_2}\)
  • Another linear combination might be: \(-1\vec{v_1} + 0\vec{v_2}\)
  • The span is the set containing ALL possible combinations: \(\{c_1\vec{v_1} + c_2\vec{v_2} \mid c_1, c_2 \in \mathbb{R}\}\)

Question: How do I determine if a vector is in the span of a set of vectors?

Answer: To determine if a vector \(\vec{b}\) is in the span of vectors \(\{\vec{v_1}, \vec{v_2}, ..., \vec{v_n}\}\), you need to:

  1. Set up the equation: \(\vec{b} = c_1\vec{v_1} + c_2\vec{v_2} + ... + c_n\vec{v_n}\)
  2. Convert this to a system of linear equations
  3. Solve for the coefficients \(c_1, c_2, ..., c_n\)
  4. If the system has a solution, \(\vec{b}\) is in the span; if no solution exists, it's not in the span

For example, to check if \(\vec{b} = \begin{pmatrix} 5 \\ 7 \end{pmatrix}\) is in the span of \(\vec{v_1} = \begin{pmatrix} 1 \\ 2 \end{pmatrix}\) and \(\vec{v_2} = \begin{pmatrix} 3 \\ 1 \end{pmatrix}\), solve: \(\begin{cases} c_1 + 3c_2 = 5 \\ 2c_1 + c_2 = 7 \end{cases}\)

Question: What's the practical significance of linear independence?

Answer: Linear independence is crucial for several reasons:

  • Minimal representation: Linearly independent vectors provide the most efficient way to represent a space without redundancy
  • Unique coordinates: In a linearly independent set, each vector has a unique representation as a linear combination
  • Basis formation: Linearly independent spanning sets form a basis, which defines the dimension of a vector space
  • Stability in computations: Linearly independent vectors are less prone to numerical errors in calculations

In practical terms, if vectors are linearly dependent, one of them provides redundant information. For example, if you're designing a coordinate system, linearly dependent vectors would create unnecessary complexity without adding new directions.

Question: How are linear combinations used in computer graphics?

Answer: Linear combinations are fundamental in computer graphics:

  • Transformations: Rotation, scaling, and translation matrices are applied using linear combinations
  • Interpolation: Bézier curves and surfaces are defined as linear combinations of control points
  • Shading: Lighting calculations involve linear combinations of light sources
  • Animation: Morphing between shapes uses linear combinations of vertex positions

For example, a point on a Bézier curve is calculated as: \(\vec{P}(t) = (1-t)^2\vec{P_0} + 2t(1-t)\vec{P_1} + t^2\vec{P_2}\), which is a linear combination of the control points with time-dependent coefficients.

Question: How do span and linear independence relate to machine learning?

Answer: These concepts are central to machine learning:

  1. Feature selection: Linear independence helps identify which features provide unique information
  2. Principal Component Analysis (PCA): Finds linearly independent directions that capture maximum variance
  3. Overfitting prevention: Linearly dependent features can lead to overfitting and numerical instability
  4. Model interpretability: Linearly independent features provide clearer insights into model behavior

For example, in PCA, we find principal components that are linearly independent and span the space of maximum variance in the data. This allows us to reduce dimensionality while preserving important information.