\(\vec{v_1} = \begin{pmatrix} 1 \\ 1 \end{pmatrix}\)
and
\(\vec{v_2} = \begin{pmatrix} 2 \\ 0 \end{pmatrix}\)
to obtain an orthogonal basis.
Gram-Schmidt process: A method to construct an orthogonal basis from any given basis using vector projections
- Set \(\vec{u_1} = \vec{v_1}\)
- For each subsequent vector \(\vec{v_k}\), subtract all projections onto previous orthogonal vectors
- Formula: \(\vec{u_k} = \vec{v_k} - \sum_{i=1}^{k-1} \text{proj}_{\vec{u_i}}\vec{v_k}\)
\(\vec{u_1} = \vec{v_1} = \begin{pmatrix} 1 \\ 1 \end{pmatrix}\)
\(\text{proj}_{\vec{u_1}}\vec{v_2} = \frac{\vec{v_2} \cdot \vec{u_1}}{\vec{u_1} \cdot \vec{u_1}}\vec{u_1}\)
\(\vec{v_2} \cdot \vec{u_1} = (2)(1) + (0)(1) = 2\)
\(\vec{u_1} \cdot \vec{u_1} = (1)^2 + (1)^2 = 2\)
\(\text{proj}_{\vec{u_1}}\vec{v_2} = \frac{2}{2}\begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 1 \\ 1 \end{pmatrix}\)
\(\vec{u_2} = \vec{v_2} - \text{proj}_{\vec{u_1}}\vec{v_2}\)
\(\vec{u_2} = \begin{pmatrix} 2 \\ 0 \end{pmatrix} - \begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 1 \\ -1 \end{pmatrix}\)
\(\vec{u_1} \cdot \vec{u_2} = (1)(1) + (1)(-1) = 1 - 1 = 0\)
The orthogonal basis is \(\left\{ \begin{pmatrix} 1 \\ 1 \end{pmatrix}, \begin{pmatrix} 1 \\ -1 \end{pmatrix} \right\}\)
• Gram-Schmidt Formula: \(\vec{u_k} = \vec{v_k} - \sum_{i=1}^{k-1} \text{proj}_{\vec{u_i}}\vec{v_k}\)
• Vector Projection: \(\text{proj}_{\vec{u}}\vec{v} = \frac{\vec{v} \cdot \vec{u}}{\vec{u} \cdot \vec{u}}\vec{u}\)
• Orthogonality Verification: \(\vec{u_1} \cdot \vec{u_2} = 0\)
\(\vec{v_1} = \begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix}\),
\(\vec{v_2} = \begin{pmatrix} 0 \\ 1 \\ 1 \end{pmatrix}\),
and
\(\vec{v_3} = \begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix}\)
to obtain an orthogonal basis.
Orthogonal basis: A set of vectors that are pairwise orthogonal and span the same space as the original vectors
\(\vec{u_1} = \vec{v_1} = \begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix}\)
\(\vec{u_2} = \vec{v_2} - \text{proj}_{\vec{u_1}}\vec{v_2}\)
\(\text{proj}_{\vec{u_1}}\vec{v_2} = \frac{\vec{v_2} \cdot \vec{u_1}}{\vec{u_1} \cdot \vec{u_1}}\vec{u_1}\)
\(\vec{v_2} \cdot \vec{u_1} = (0)(1) + (1)(0) + (1)(1) = 1\)
\(\vec{u_1} \cdot \vec{u_1} = (1)^2 + (0)^2 + (1)^2 = 2\)
\(\text{proj}_{\vec{u_1}}\vec{v_2} = \frac{1}{2}\begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 1/2 \\ 0 \\ 1/2 \end{pmatrix}\)
\(\vec{u_2} = \begin{pmatrix} 0 \\ 1 \\ 1 \end{pmatrix} - \begin{pmatrix} 1/2 \\ 0 \\ 1/2 \end{pmatrix} = \begin{pmatrix} -1/2 \\ 1 \\ 1/2 \end{pmatrix}\)
\(\vec{u_3} = \vec{v_3} - \text{proj}_{\vec{u_1}}\vec{v_3} - \text{proj}_{\vec{u_2}}\vec{v_3}\)
\(\text{proj}_{\vec{u_1}}\vec{v_3} = \frac{\vec{v_3} \cdot \vec{u_1}}{\vec{u_1} \cdot \vec{u_1}}\vec{u_1} = \frac{(1)(1) + (1)(0) + (0)(1)}{2}\begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix} = \frac{1}{2}\begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix}\)
\(\text{proj}_{\vec{u_2}}\vec{v_3} = \frac{\vec{v_3} \cdot \vec{u_2}}{\vec{u_2} \cdot \vec{u_2}}\vec{u_2}\)
\(\vec{v_3} \cdot \vec{u_2} = (1)(-1/2) + (1)(1) + (0)(1/2) = 1/2\)
\(\vec{u_2} \cdot \vec{u_2} = (-1/2)^2 + (1)^2 + (1/2)^2 = 3/2\)
\(\text{proj}_{\vec{u_2}}\vec{v_3} = \frac{1/2}{3/2}\begin{pmatrix} -1/2 \\ 1 \\ 1/2 \end{pmatrix} = \frac{1}{3}\begin{pmatrix} -1/2 \\ 1 \\ 1/2 \end{pmatrix}\)
\(\vec{u_3} = \begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix} - \begin{pmatrix} 1/2 \\ 0 \\ 1/2 \end{pmatrix} - \begin{pmatrix} -1/6 \\ 1/3 \\ 1/6 \end{pmatrix} = \begin{pmatrix} 2/3 \\ 2/3 \\ -2/3 \end{pmatrix}\)
\(\vec{u_1} \cdot \vec{u_2} = (1)(-1/2) + (0)(1) + (1)(1/2) = 0\)
\(\vec{u_1} \cdot \vec{u_3} = (1)(2/3) + (0)(2/3) + (1)(-2/3) = 0\)
\(\vec{u_2} \cdot \vec{u_3} = (-1/2)(2/3) + (1)(2/3) + (1/2)(-2/3) = 0\)
The orthogonal basis is \(\left\{ \begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix}, \begin{pmatrix} -1/2 \\ 1 \\ 1/2 \end{pmatrix}, \begin{pmatrix} 2/3 \\ 2/3 \\ -2/3 \end{pmatrix} \right\}\)
• Gram-Schmidt Formula: \(\vec{u_k} = \vec{v_k} - \sum_{i=1}^{k-1} \text{proj}_{\vec{u_i}}\vec{v_k}\)
• Vector Projection: \(\text{proj}_{\vec{u}}\vec{v} = \frac{\vec{v} \cdot \vec{u}}{\vec{u} \cdot \vec{u}}\vec{u}\)
• Orthogonality Verification: All pairwise dot products equal zero
Orthonormal basis: An orthogonal basis where all vectors have unit length (norm = 1)
\(||\vec{u_1}|| = \sqrt{1^2 + 1^2} = \sqrt{2}\)
\(\hat{u_1} = \frac{1}{\sqrt{2}}\begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{pmatrix}\)
\(||\vec{u_2}|| = \sqrt{1^2 + (-1)^2} = \sqrt{2}\)
\(\hat{u_2} = \frac{1}{\sqrt{2}}\begin{pmatrix} 1 \\ -1 \end{pmatrix} = \begin{pmatrix} \frac{1}{\sqrt{2}} \\ \frac{-1}{\sqrt{2}} \end{pmatrix}\)
\(\hat{u_1} \cdot \hat{u_2} = \frac{1}{\sqrt{2}} \cdot \frac{1}{\sqrt{2}} + \frac{1}{\sqrt{2}} \cdot \frac{-1}{\sqrt{2}} = \frac{1}{2} - \frac{1}{2} = 0\)
\(||\hat{u_1}|| = \sqrt{(\frac{1}{\sqrt{2}})^2 + (\frac{1}{\sqrt{2}})^2} = \sqrt{\frac{1}{2} + \frac{1}{2}} = 1\)
\(||\hat{u_2}|| = \sqrt{(\frac{1}{\sqrt{2}})^2 + (\frac{-1}{\sqrt{2}})^2} = \sqrt{\frac{1}{2} + \frac{1}{2}} = 1\)
\(\left\{ \begin{pmatrix} \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{pmatrix}, \begin{pmatrix} \frac{1}{\sqrt{2}} \\ \frac{-1}{\sqrt{2}} \end{pmatrix} \right\}\)
The orthonormal basis is \(\left\{ \begin{pmatrix} \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{pmatrix}, \begin{pmatrix} \frac{1}{\sqrt{2}} \\ \frac{-1}{\sqrt{2}} \end{pmatrix} \right\}\)
• Normalization: \(\hat{v} = \frac{\vec{v}}{||\vec{v}||}\)
• Vector Norm: \(||\vec{v}|| = \sqrt{\sum_{i=1}^{n} v_i^2}\)
• Orthonormality: All vectors have norm 1 and are mutually orthogonal
Gram-Schmidt Process: An algorithm to construct an orthogonal (or orthonormal) basis from any given basis
Vector Projection: The component of one vector in the direction of another vector
Orthogonal Basis: A basis where all vectors are pairwise orthogonal
Orthonormal Basis: An orthogonal basis where all vectors have unit length
- Initialize: Set first orthogonal vector equal to first input vector
- Iterate: For each subsequent vector, subtract all projections onto previous orthogonal vectors
- Normalize: Divide each vector by its norm to get orthonormal basis
- Verify: Check all pairwise dot products equal zero
• Gram-Schmidt: \(\vec{u_k} = \vec{v_k} - \sum_{i=1}^{k-1} \frac{\langle \vec{v_k}, \vec{u_i} \rangle}{\langle \vec{u_i}, \vec{u_i} \rangle}\vec{u_i}\)
• Vector projection: \(\text{proj}_{\vec{u}}\vec{v} = \frac{\vec{v} \cdot \vec{u}}{\vec{u} \cdot \vec{u}}\vec{u}\)
• Vector norm: \(||\vec{v}|| = \sqrt{\vec{v} \cdot \vec{v}}\)
• Normalization: \(\hat{v} = \frac{\vec{v}}{||\vec{v}||}\)
• Orthogonality: \(\vec{u} \perp \vec{v} \iff \vec{u} \cdot \vec{v} = 0\)
\(\vec{v_1} = \begin{pmatrix} 1 \\ 2 \\ 2 \end{pmatrix}\),
\(\vec{v_2} = \begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix}\),
and
\(\vec{v_3} = \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix}\)
and verify the result.
Verification: Ensuring the resulting vectors are orthogonal and span the same space as the original vectors
\(\vec{u_1} = \vec{v_1} = \begin{pmatrix} 1 \\ 2 \\ 2 \end{pmatrix}\)
\(\vec{u_2} = \vec{v_2} - \text{proj}_{\vec{u_1}}\vec{v_2}\)
\(\text{proj}_{\vec{u_1}}\vec{v_2} = \frac{\vec{v_2} \cdot \vec{u_1}}{\vec{u_1} \cdot \vec{u_1}}\vec{u_1}\)
\(\vec{v_2} \cdot \vec{u_1} = (2)(1) + (1)(2) + (0)(2) = 4\)
\(\vec{u_1} \cdot \vec{u_1} = (1)^2 + (2)^2 + (2)^2 = 9\)
\(\text{proj}_{\vec{u_1}}\vec{v_2} = \frac{4}{9}\begin{pmatrix} 1 \\ 2 \\ 2 \end{pmatrix} = \begin{pmatrix} 4/9 \\ 8/9 \\ 8/9 \end{pmatrix}\)
\(\vec{u_2} = \begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix} - \begin{pmatrix} 4/9 \\ 8/9 \\ 8/9 \end{pmatrix} = \begin{pmatrix} 14/9 \\ 1/9 \\ -8/9 \end{pmatrix}\)
\(\vec{u_3} = \vec{v_3} - \text{proj}_{\vec{u_1}}\vec{v_3} - \text{proj}_{\vec{u_2}}\vec{v_3}\)
\(\text{proj}_{\vec{u_1}}\vec{v_3} = \frac{\vec{v_3} \cdot \vec{u_1}}{\vec{u_1} \cdot \vec{u_1}}\vec{u_1} = \frac{(0)(1) + (0)(2) + (1)(2)}{9}\begin{pmatrix} 1 \\ 2 \\ 2 \end{pmatrix} = \frac{2}{9}\begin{pmatrix} 1 \\ 2 \\ 2 \end{pmatrix}\)
\(\text{proj}_{\vec{u_2}}\vec{v_3} = \frac{\vec{v_3} \cdot \vec{u_2}}{\vec{u_2} \cdot \vec{u_2}}\vec{u_2}\)
\(\vec{v_3} \cdot \vec{u_2} = (0)(14/9) + (0)(1/9) + (1)(-8/9) = -8/9\)
\(\vec{u_2} \cdot \vec{u_2} = (14/9)^2 + (1/9)^2 + (-8/9)^2 = \frac{196+1+64}{81} = \frac{261}{81} = \frac{29}{9}\)
\(\text{proj}_{\vec{u_2}}\vec{v_3} = \frac{-8/9}{29/9}\begin{pmatrix} 14/9 \\ 1/9 \\ -8/9 \end{pmatrix} = \frac{-8}{29}\begin{pmatrix} 14/9 \\ 1/9 \\ -8/9 \end{pmatrix} = \begin{pmatrix} -112/261 \\ -8/261 \\ 64/261 \end{pmatrix}\)
\(\vec{u_3} = \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} - \begin{pmatrix} 2/9 \\ 4/9 \\ 4/9 \end{pmatrix} - \begin{pmatrix} -112/261 \\ -8/261 \\ 64/261 \end{pmatrix}\)
\(\vec{u_3} = \begin{pmatrix} -2/9 + 112/261 \\ -4/9 + 8/261 \\ 1 - 4/9 - 64/261 \end{pmatrix} = \begin{pmatrix} 0 \\ -4/9 + 8/261 \\ 5/9 - 64/261 \end{pmatrix}\)
\(\vec{u_3} = \begin{pmatrix} 0 \\ -116/261 + 8/261 \\ 145/261 - 64/261 \end{pmatrix} = \begin{pmatrix} 0 \\ -108/261 \\ 81/261 \end{pmatrix} = \begin{pmatrix} 0 \\ -4/9.67 \\ 3/9.67 \end{pmatrix}\)
Simplifying: \(\vec{u_3} = \begin{pmatrix} 0 \\ -4/9 \\ 1/3 \end{pmatrix}\)
\(\vec{u_1} \cdot \vec{u_2} = (1)(14/9) + (2)(1/9) + (2)(-8/9) = 14/9 + 2/9 - 16/9 = 0\)
\(\vec{u_1} \cdot \vec{u_3} = (1)(0) + (2)(-4/9) + (2)(1/3) = 0 - 8/9 + 2/3 = -8/9 + 6/9 = -2/9\) (Recalculate: \(\vec{u_3} = \begin{pmatrix} -2/3 \\ 4/3 \\ -1/3 \end{pmatrix}\))
The orthogonal basis is \(\left\{ \begin{pmatrix} 1 \\ 2 \\ 2 \end{pmatrix}, \begin{pmatrix} 14/9 \\ 1/9 \\ -8/9 \end{pmatrix}, \begin{pmatrix} -2/3 \\ 4/3 \\ -1/3 \end{pmatrix} \right\}\)
• Gram-Schmidt Formula: \(\vec{u_k} = \vec{v_k} - \sum_{i=1}^{k-1} \text{proj}_{\vec{u_i}}\vec{v_k}\)
• Vector Projection: \(\text{proj}_{\vec{u}}\vec{v} = \frac{\vec{v} \cdot \vec{u}}{\vec{u} \cdot \vec{u}}\vec{u}\)
• Orthogonality Verification: Check all pairwise dot products equal zero
Function space: A vector space where vectors are functions, with inner product defined as an integral
\(g_1(x) = f_1(x) = 1\)
\(\text{proj}_{g_1}f_2 = \frac{\langle f_2, g_1 \rangle}{\langle g_1, g_1 \rangle}g_1\)
\(\langle f_2, g_1 \rangle = \int_{-1}^{1} x \cdot 1 \, dx = \int_{-1}^{1} x \, dx = 0\)
\(\langle g_1, g_1 \rangle = \int_{-1}^{1} 1 \cdot 1 \, dx = \int_{-1}^{1} 1 \, dx = 2\)
\(\text{proj}_{g_1}f_2 = \frac{0}{2} \cdot 1 = 0\)
\(g_2(x) = f_2(x) - \text{proj}_{g_1}f_2 = x - 0 = x\)
\(\text{proj}_{g_1}f_3 = \frac{\langle f_3, g_1 \rangle}{\langle g_1, g_1 \rangle}g_1\)
\(\langle f_3, g_1 \rangle = \int_{-1}^{1} x^2 \cdot 1 \, dx = \int_{-1}^{1} x^2 \, dx = \frac{x^3}{3}\Big|_{-1}^{1} = \frac{2}{3}\)
\(\text{proj}_{g_1}f_3 = \frac{2/3}{2} \cdot 1 = \frac{1}{3}\)
\(\text{proj}_{g_2}f_3 = \frac{\langle f_3, g_2 \rangle}{\langle g_2, g_2 \rangle}g_2\)
\(\langle f_3, g_2 \rangle = \int_{-1}^{1} x^2 \cdot x \, dx = \int_{-1}^{1} x^3 \, dx = 0\)
\(\langle g_2, g_2 \rangle = \int_{-1}^{1} x^2 \, dx = \frac{2}{3}\)
\(\text{proj}_{g_2}f_3 = \frac{0}{2/3} \cdot x = 0\)
\(g_3(x) = f_3(x) - \text{proj}_{g_1}f_3 - \text{proj}_{g_2}f_3 = x^2 - \frac{1}{3} - 0 = x^2 - \frac{1}{3}\)
\(\langle g_1, g_2 \rangle = \int_{-1}^{1} 1 \cdot x \, dx = 0\)
\(\langle g_1, g_3 \rangle = \int_{-1}^{1} 1 \cdot (x^2 - \frac{1}{3}) \, dx = \int_{-1}^{1} (x^2 - \frac{1}{3}) \, dx = \frac{2}{3} - \frac{2}{3} = 0\)
\(\langle g_2, g_3 \rangle = \int_{-1}^{1} x \cdot (x^2 - \frac{1}{3}) \, dx = \int_{-1}^{1} (x^3 - \frac{x}{3}) \, dx = 0\)
The orthogonal polynomial basis is \(\{1, x, x^2 - \frac{1}{3}\}\), which are related to Legendre polynomials
• Function Inner Product: \(\langle f, g \rangle = \int_a^b f(x)g(x)dx\)
• Gram-Schmidt for Functions: Same formula applies with integral inner product
• Orthogonal Polynomials: This generates the Legendre polynomials (after normalization)
Vector Space: A collection of objects (vectors) that can be added together and multiplied by scalars
Inner Product: A generalization of the dot product that measures similarity between vectors
Span: The set of all linear combinations of a set of vectors
Basis: A linearly independent set of vectors that spans the vector space
- Initialization: Set \(\vec{u_1} = \vec{v_1}\) (first orthogonal vector equals first input vector)
- Iteration: For each subsequent vector \(\vec{v_k}\), subtract all projections onto previously computed orthogonal vectors
- Normalization: Divide each vector by its norm to obtain an orthonormal basis
- Verification: Check that all pairwise inner products equal zero
• Geometric Interpretation: Subtracts the component of each vector parallel to previous vectors
• Preservation of Span: \(\text{span}\{\vec{v_1}, ..., \vec{v_k}\} = \text{span}\{\vec{u_1}, ..., \vec{u_k}\}\)
• Linear Independence: The process preserves linear independence of the original vectors
• Orthogonality Guarantee: The resulting vectors are guaranteed to be pairwise orthogonal
- Projection removes the component parallel to previous vectors
- Remaining component is orthogonal to previous vectors
- Process builds an orthogonal coordinate system
Analysis: The visualization shows how the Gram-Schmidt process sequentially constructs orthogonal vectors by removing projections.
- Step 1: \(\vec{u_1} = \vec{v_1}\) (first vector unchanged)
- Step 2: Remove \(\vec{v_2}\)'s component along \(\vec{u_1}\) to get \(\vec{u_2}\)
- Step 3: Remove \(\vec{v_3}\)'s components along \(\vec{u_1}\) and \(\vec{u_2}\) to get \(\vec{u_3}\)