Vector Projections and Orthogonal Decomposition - Comprehensive Summary
Vector Projection: \(\text{proj}_{\vec{u}}\vec{v} = \frac{\vec{v} \cdot \vec{u}}{\vec{u} \cdot \vec{u}}\vec{u}\), the component of \(\vec{v}\) in the direction of \(\vec{u}\)
Orthogonal Decomposition: Expressing \(\vec{v}\) as \(\vec{v} = \text{proj}_{\vec{u}}\vec{v} + \vec{v}_\perp\) where \(\vec{v}_\perp \perp \vec{u}\)
Gram-Schmidt Process: Method to construct orthogonal/orthonormal basis from any set of vectors
• Projection is in direction of reference vector
• Perpendicular component is orthogonal to reference
• Orthogonal vectors have zero dot product
• Normalized vectors have unit length
1. Calculate projection using dot product formula
2. Find perpendicular component by subtraction
3. Apply Gram-Schmidt iteratively for multiple vectors
4. Normalize vectors for orthonormal basis
• Always check that \(\vec{v}_\perp \cdot \vec{u} = 0\)
• Projection magnitude ≤ original vector magnitude
• Use geometric intuition to verify results
• Order matters in Gram-Schmidt process
Vector Projection: \(\text{proj}_{\vec{u}}\vec{v} = \frac{\vec{v} \cdot \vec{u}}{\vec{u} \cdot \vec{u}}\vec{u}\). This gives the component of \(\vec{v}\) that lies in the direction of \(\vec{u}\).
- Calculate \(\vec{v} \cdot \vec{u}\) (numerator of fraction)
- Calculate \(\vec{u} \cdot \vec{u}\) (denominator of fraction)
- Compute the scalar \(\frac{\vec{v} \cdot \vec{u}}{\vec{u} \cdot \vec{u}}\)
- Multiply this scalar by vector \(\vec{u}\) to get the projection
- Find the orthogonal component: \(\vec{v}_\perp = \vec{v} - \text{proj}_{\vec{u}}\vec{v}\)
\(\vec{v} \cdot \vec{u} = (3)(1) + (4)(2) = 3 + 8 = 11\)
\(\vec{u} \cdot \vec{u} = (1)(1) + (2)(2) = 1 + 4 = 5\)
\(\frac{\vec{v} \cdot \vec{u}}{\vec{u} \cdot \vec{u}} = \frac{11}{5}\)
\(\text{proj}_{\vec{u}}\vec{v} = \frac{11}{5}\vec{u} = \frac{11}{5}\begin{pmatrix} 1 \\ 2 \end{pmatrix} = \begin{pmatrix} \frac{11}{5} \\ \frac{22}{5} \end{pmatrix}\)
\(\vec{v}_\perp = \vec{v} - \text{proj}_{\vec{u}}\vec{v} = \begin{pmatrix} 3 \\ 4 \end{pmatrix} - \begin{pmatrix} \frac{11}{5} \\ \frac{22}{5} \end{pmatrix} = \begin{pmatrix} 3 - \frac{11}{5} \\ 4 - \frac{22}{5} \end{pmatrix} = \begin{pmatrix} \frac{15-11}{5} \\ \frac{20-22}{5} \end{pmatrix} = \begin{pmatrix} \frac{4}{5} \\ \frac{-2}{5} \end{pmatrix}\)
The projection of \(\vec{v}\) onto \(\vec{u}\) is \(\begin{pmatrix} \frac{11}{5} \\ \frac{22}{5} \end{pmatrix}\) and the orthogonal component is \(\begin{pmatrix} \frac{4}{5} \\ \frac{-2}{5} \end{pmatrix}\).
• Projection formula: Use dot product ratio multiplied by reference vector
• Orthogonal decomposition: \(\vec{v} = \text{proj}_{\vec{u}}\vec{v} + \vec{v}_\perp\)
• Verification: Check that \(\vec{v}_\perp \cdot \vec{u} = 0\)
Orthogonal Decomposition: Express vector \(\vec{w}\) as \(\vec{w} = \vec{w}_{\parallel} + \vec{w}_{\perp}\) where \(\vec{w}_{\parallel}\) is parallel to \(\vec{n}\) and \(\vec{w}_{\perp}\) is orthogonal to \(\vec{n}\).
\(\vec{w} \cdot \vec{n} = (5)(2) + (1)(1) + (3)(0) = 10 + 1 + 0 = 11\)
\(\vec{n} \cdot \vec{n} = (2)(2) + (1)(1) + (0)(0) = 4 + 1 + 0 = 5\)
\(\vec{w}_{\parallel} = \text{proj}_{\vec{n}}\vec{w} = \frac{\vec{w} \cdot \vec{n}}{\vec{n} \cdot \vec{n}}\vec{n} = \frac{11}{5}\begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix} = \begin{pmatrix} \frac{22}{5} \\ \frac{11}{5} \\ 0 \end{pmatrix} = \begin{pmatrix} 4.4 \\ 2.2 \\ 0 \end{pmatrix}\)
\(\vec{w}_{\perp} = \vec{w} - \vec{w}_{\parallel} = \begin{pmatrix} 5 \\ 1 \\ 3 \end{pmatrix} - \begin{pmatrix} 4.4 \\ 2.2 \\ 0 \end{pmatrix} = \begin{pmatrix} 0.6 \\ -1.2 \\ 3 \end{pmatrix}\)
Check that \(\vec{w}_{\perp} \cdot \vec{n} = (0.6)(2) + (-1.2)(1) + (3)(0) = 1.2 - 1.2 + 0 = 0\)
And \(\vec{w}_{\parallel} + \vec{w}_{\perp} = \begin{pmatrix} 4.4 \\ 2.2 \\ 0 \end{pmatrix} + \begin{pmatrix} 0.6 \\ -1.2 \\ 3 \end{pmatrix} = \begin{pmatrix} 5 \\ 1 \\ 3 \end{pmatrix} = \vec{w}\)
The parallel component is \(\vec{w}_{\parallel} = \begin{pmatrix} 4.4 \\ 2.2 \\ 0 \end{pmatrix}\) and the orthogonal component is \(\vec{w}_{\perp} = \begin{pmatrix} 0.6 \\ -1.2 \\ 3 \end{pmatrix}\).
• Decomposition principle: Any vector can be split into parallel and perpendicular parts
• Orthogonality verification: Dot product of perpendicular part with reference should be zero
• Reconstruction: Sum of components equals original vector
Distance from Point to Line: The shortest distance is the length of the component of \(\overrightarrow{OP}\) that is orthogonal to the direction vector \(\vec{d}\). This equals \(\|\vec{v}_\perp\|\) where \(\vec{v}_\perp = \vec{p} - \text{proj}_{\vec{d}}\vec{p}\).
\(\vec{p} = \overrightarrow{OP} = \begin{pmatrix} 4 \\ 5 \end{pmatrix}\)
\(\vec{p} \cdot \vec{d} = (4)(3) + (5)(4) = 12 + 20 = 32\)
\(\vec{d} \cdot \vec{d} = (3)(3) + (4)(4) = 9 + 16 = 25\)
\(\text{proj}_{\vec{d}}\vec{p} = \frac{\vec{p} \cdot \vec{d}}{\vec{d} \cdot \vec{d}}\vec{d} = \frac{32}{25}\begin{pmatrix} 3 \\ 4 \end{pmatrix} = \begin{pmatrix} \frac{96}{25} \\ \frac{128}{25} \end{pmatrix} = \begin{pmatrix} 3.84 \\ 5.12 \end{pmatrix}\)
\(\vec{p}_\perp = \vec{p} - \text{proj}_{\vec{d}}\vec{p} = \begin{pmatrix} 4 \\ 5 \end{pmatrix} - \begin{pmatrix} 3.84 \\ 5.12 \end{pmatrix} = \begin{pmatrix} 0.16 \\ -0.12 \end{pmatrix}\)
Distance = \(\|\vec{p}_\perp\| = \sqrt{(0.16)^2 + (-0.12)^2} = \sqrt{0.0256 + 0.0144} = \sqrt{0.04} = 0.2\)
Wait, let's recalculate more precisely: \(\vec{p}_\perp = \begin{pmatrix} 4 \\ 5 \end{pmatrix} - \begin{pmatrix} \frac{96}{25} \\ \frac{128}{25} \end{pmatrix} = \begin{pmatrix} \frac{100-96}{25} \\ \frac{125-128}{25} \end{pmatrix} = \begin{pmatrix} \frac{4}{25} \\ \frac{-3}{25} \end{pmatrix}\)
Distance = \(\sqrt{(\frac{4}{25})^2 + (\frac{-3}{25})^2} = \sqrt{\frac{16 + 9}{625}} = \sqrt{\frac{25}{625}} = \sqrt{\frac{1}{25}} = \frac{1}{5}\)
The distance from point P(4, 5) to the line is \(\frac{1}{5}\) units.
• Shortest distance: Perpendicular distance is shortest to line
• Orthogonal projection: Gives closest point on line
• Distance calculation: Magnitude of orthogonal component
Gram-Schmidt Process: An algorithm to convert a set of linearly independent vectors into an orthogonal set. For two vectors: \(\vec{u_1} = \vec{v_1}\) and \(\vec{u_2} = \vec{v_2} - \text{proj}_{\vec{u_1}}\vec{v_2}\).
\(\vec{u_1} = \vec{v_1} = \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}\)
\(\vec{v_2} \cdot \vec{u_1} = (2)(1) + (0)(1) + (1)(1) = 2 + 0 + 1 = 3\)
\(\vec{u_1} \cdot \vec{u_1} = (1)(1) + (1)(1) + (1)(1) = 3\)
\(\text{proj}_{\vec{u_1}}\vec{v_2} = \frac{3}{3}\vec{u_1} = \vec{u_1} = \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}\)
\(\vec{u_2} = \vec{v_2} - \text{proj}_{\vec{u_1}}\vec{v_2} = \begin{pmatrix} 2 \\ 0 \\ 1 \end{pmatrix} - \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 1 \\ -1 \\ 0 \end{pmatrix}\)
\(\vec{u_1} \cdot \vec{u_2} = (1)(1) + (1)(-1) + (1)(0) = 1 - 1 + 0 = 0\)
The orthogonal basis is \(\left\{\begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}, \begin{pmatrix} 1 \\ -1 \\ 0 \end{pmatrix}\right\}\).
The orthogonal basis obtained from Gram-Schmidt process is \(\left\{\begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}, \begin{pmatrix} 1 \\ -1 \\ 0 \end{pmatrix}\right\}\).
• Gram-Schmidt algorithm: Remove components parallel to previous vectors
• Orthogonality verification: Dot products between different vectors should be zero
• Span preservation: New vectors span the same space as original
Orthonormal Basis: A basis where all vectors are orthogonal (perpendicular) and normalized (unit length). Each vector has magnitude 1 and dot products between different vectors are 0.
\(\|\vec{u_1}\| = \sqrt{1^2 + 1^2 + 1^2} = \sqrt{3}\)
\(\vec{e_1} = \frac{\vec{u_1}}{\|\vec{u_1}\|} = \frac{1}{\sqrt{3}}\begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix} = \begin{pmatrix} \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \end{pmatrix}\)
\(\|\vec{u_2}\| = \sqrt{1^2 + (-1)^2 + 0^2} = \sqrt{2}\)
\(\vec{e_2} = \frac{\vec{u_2}}{\|\vec{u_2}\|} = \frac{1}{\sqrt{2}}\begin{pmatrix} 1 \\ -1 \\ 0 \end{pmatrix} = \begin{pmatrix} \frac{1}{\sqrt{2}} \\ \frac{-1}{\sqrt{2}} \\ 0 \end{pmatrix}\)
\(\vec{e_1} \cdot \vec{e_2} = \frac{1}{\sqrt{3}} \cdot \frac{1}{\sqrt{2}} + \frac{1}{\sqrt{3}} \cdot \frac{-1}{\sqrt{2}} + \frac{1}{\sqrt{3}} \cdot 0 = \frac{1}{\sqrt{6}} - \frac{1}{\sqrt{6}} + 0 = 0\)
\(\|\vec{e_1}\| = \sqrt{\left(\frac{1}{\sqrt{3}}\right)^2 + \left(\frac{1}{\sqrt{3}}\right)^2 + \left(\frac{1}{\sqrt{3}}\right)^2} = \sqrt{\frac{1}{3} + \frac{1}{3} + \frac{1}{3}} = \sqrt{1} = 1\)
\(\|\vec{e_2}\| = \sqrt{\left(\frac{1}{\sqrt{2}}\right)^2 + \left(\frac{-1}{\sqrt{2}}\right)^2 + 0^2} = \sqrt{\frac{1}{2} + \frac{1}{2}} = \sqrt{1} = 1\)
The orthonormal basis is \(\left\{\begin{pmatrix} \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \end{pmatrix}, \begin{pmatrix} \frac{1}{\sqrt{2}} \\ \frac{-1}{\sqrt{2}} \\ 0 \end{pmatrix}\right\}\).
The orthonormal basis is \(\left\{\begin{pmatrix} \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \end{pmatrix}, \begin{pmatrix} \frac{1}{\sqrt{2}} \\ \frac{-1}{\sqrt{2}} \\ 0 \end{pmatrix}\right\}\).
• Normalization: Divide vector by its magnitude to get unit vector
• Orthonormal verification: All vectors have unit length and are mutually orthogonal
• Preservation of span: Orthonormal basis spans same space as original
Vector Projection: The component of one vector that lies in the direction of another vector. Measures "how much" of the first vector points in the direction of the second.
Orthogonal Decomposition: Breaking a vector into two components: one parallel to a reference vector and one perpendicular to it.
Gram-Schmidt Process: A method for generating an orthogonal (or orthonormal) basis from a given set of vectors.
- For projections: Use the dot product formula with the reference vector
- For orthogonal decomposition: Subtract the projection from the original vector
- For Gram-Schmidt: Iteratively subtract projections onto previous vectors
- For normalization: Divide each vector by its magnitude
• \(\text{proj}_{\vec{u}}\vec{v} = \text{proj}_{\vec{u}}(c\vec{v})\) for scalar \(c\)
• \(\text{proj}_{\vec{u}}(\vec{v_1} + \vec{v_2}) = \text{proj}_{\vec{u}}\vec{v_1} + \text{proj}_{\vec{u}}\vec{v_2}\)
• \(\text{proj}_{\vec{u}}\vec{u} = \vec{u}\) (self-projection)
• Pythagorean theorem: \(\|\vec{v}\|^2 = \|\text{proj}_{\vec{u}}\vec{v}\|^2 + \|\vec{v}_\perp\|^2\)
• The projection of \(\vec{v}\) onto \(\vec{u}\) has the same direction as \(\vec{u}\)
Geometric Interpretation: The projection of \(\vec{v}\) onto \(\vec{u}\) is the "shadow" of \(\vec{v}\) when light shines perpendicular to \(\vec{u}\). The orthogonal component completes the right triangle.
- \(\vec{v} = \begin{pmatrix} 4 \\ 3 \end{pmatrix}\) (original vector)
- \(\vec{u} = \begin{pmatrix} 3 \\ 0 \end{pmatrix}\) (reference vector)
- \(\text{proj}_{\vec{u}}\vec{v} = \begin{pmatrix} 4 \\ 0 \end{pmatrix}\) (projection)
- \(\vec{v}_\perp = \begin{pmatrix} 0 \\ 3 \end{pmatrix}\) (orthogonal component)