Change of Basis: Transforming vector coordinates from one basis to another
Transition Matrix: P_{B←C} converts coordinates from basis C to basis B
Principal Component Analysis (PCA): Change to basis of eigenvectors to capture maximum variance
Fourier Analysis: Change to frequency domain basis using sine/cosine functions
- Identify bases: Original basis B and new basis C
- Form transition matrix: P_{B←C} with columns being C-basis vectors in B-coordinates
- Apply transformation: [v]_B = P_{B←C}[v]_C
- Verify: Check that vectors represent the same geometric entity
• Change of basis: [v]_B = P_{B←C}[v]_C
• Transition matrix: P_{B←C} = [ [c₁]_B [c₂]_B ... [cₙ]_B ]
• PCA: Covariance matrix C = (1/n)X^T X, eigendecomposition C = VΛV^T
• Fourier series: f(t) = a₀/2 + Σ(aₙcos(nt) + bₙsin(nt))
Transition Matrix: Converts coordinates from basis C to basis B: [v]_B = P_{B←C}[v]_C
- Express each C-basis vector in B-coordinates
- Form transition matrix with these as columns
- Apply the transformation to convert coordinates
c₁ = [2, 1] in standard basis → [c₁]_B =
c₂ = [1, 3] in standard basis → [c₂]_B =
P_{B←C} = [ [c₁]_B [c₂]_B ] =
[v]_B = P_{B←C}[v]_C =
=
Original vector in C-coordinates: v = 3c₁ + 2c₂ = 3[2, 1] + 2[1, 3] = [6, 3] + [2, 6] = [8, 9] ✓
Vector [v]_C = [3, 2] in basis C equals [v]_B = [8, 9] in standard basis
• Transition matrix: Columns are new basis vectors in old coordinates
• Matrix multiplication: Row by column rule
• Verification: Check that vector representation is consistent
Principal Component Analysis: Change to basis that maximizes variance along principal components
Mean of each feature: μ₁ = (1+2+3+4)/4 = 2.5, μ₂ = (2+3+4+5)/4 = 3.5
X_centered =
C = (1/n)X_centered^T X_centered = (1/4)
= (1/4)
= (1/4)
det(C - λI) = det
(1.25-λ)² = (1.25)² → 1.25-λ = ±1.25 → λ₁ = 2.5, λ₂ = 0
For λ₁ = 2.5: (C - 2.5I)v = 0 →
This gives: -1.25x + 1.25y = 0 → y = x, so v₁ =
For λ₂ = 0: (C)v = 0 → Similar process gives v₂ =
First principal component captures 100% of the variance (data lies on a line)
• Data centering: Subtract mean from each feature
• Covariance matrix: Measures relationships between features
• Eigendecomposition: Finds principal directions of maximum variance
Fourier Series: Representation of periodic function using sines and cosines as basis functions
a₀ = ⟨f(t), 1/√(2π)⟩ = (1/√(2π)) ∫₋π^π t dt
= (1/√(2π)) [t²/2]₋π^π = (1/√(2π)) [π²/2 - π²/2] = 0
aₙ = ⟨f(t), cos(nt)/√π⟩ = (1/√π) ∫₋π^π t cos(nt) dt
Using integration by parts: u = t, dv = cos(nt)dt → du = dt, v = sin(nt)/n
= (1/√π) [t sin(nt)/n]₋π^π - (1/√π) ∫₋π^π sin(nt)/n dt
= (1/√π) [π sin(nπ)/n - (-π) sin(-nπ)/n] - (1/√π) [-cos(nt)/(n²)]₋π^π
= (1/√π) [0 - 0] - (1/√π) [-cos(nπ)/(n²) - (-cos(-nπ))/(n²)]
= (1/√π) [cos(nπ)/(n²) - cos(nπ)/(n²)] = 0
Wait, let's recalculate more carefully: ∫₋π^π t cos(nt) dt = [t sin(nt)/n]₋π^π - ∫₋π^π sin(nt)/n dt
= π sin(nπ)/n - (-π) sin(-nπ)/n - [−cos(nt)/(n²)]₋π^π
= 0 - 0 - [−cos(nπ)/(n²) + cos(−nπ)/(n²)] = 0
bₙ = ⟨f(t), sin(nt)/√π⟩ = (1/√π) ∫₋π^π t sin(nt) dt
Using integration by parts: u = t, dv = sin(nt)dt → du = dt, v = -cos(nt)/n
= (1/√π) [-t cos(nt)/n]₋π^π + (1/√π) ∫₋π^π cos(nt)/n dt
= (1/√π) [-π cos(nπ)/n - (-(-π) cos(-nπ)/n)] + (1/√π) [sin(nt)/(n²)]₋π^π
= (1/√π) [-π cos(nπ)/n - π cos(nπ)/n] + (1/√π) [0 - 0]
= (1/√π) [-2π cos(nπ)/n] = -2π cos(nπ)/(n√π) = -2√π cos(nπ)/n
Since cos(nπ) = (-1)ⁿ, we have: bₙ = -2√π (-1)ⁿ/n = 2√π (-1)ⁿ⁺¹/n
f(t) = Σ bₙ sin(nt)/√π = Σ [2√π (-1)ⁿ⁺¹/n] × sin(nt)/√π = Σ 2(-1)ⁿ⁺¹ sin(nt)/n
= 2[sin(t) - sin(2t)/2 + sin(3t)/3 - sin(4t)/4 + ...]
Only sine terms appear (since f(t) = t is odd), with coefficients bₙ = 2(-1)ⁿ⁺¹/n
• Inner product: ⟨f,g⟩ = ∫f(t)g(t)dt for function spaces
• Orthonormal basis: Functions with ⟨φᵢ,φⱼ⟩ = δᵢⱼ
• Integration by parts: Essential for computing Fourier coefficients
PCA Coordinate Transformation: Change to basis of principal components to reduce dimensionality
PC₁ =
Original point: [3, 4]
Centered point: [3, 4] - [2.5, 3.5] = [0.5, 0.5]
Coordinate in PC₁ direction: ⟨[0.5, 0.5], PC₁⟩ = [0.5, 0.5] ·
Coordinate in PC₂ direction: ⟨[0.5, 0.5], PC₂⟩ = [0.5, 0.5] ·
The transformation matrix from standard coordinates to PCA coordinates is:
P_{PCA←std} =
Wait, this is actually P_{std←PCA}. The correct transformation is P_{PCA←std} = (P_{std←PCA})^T =
[v]_{PCA} = P_{PCA←std}[v]_{std} =
=
Point [3, 4] projects to [1/√2, 0] in PCA coordinate system (along PC₁ direction)
• Data centering: Always center data before PCA transformation
• Projection: Inner product measures component along basis vector
• Dimensionality reduction: Small components can be ignored
Discrete Fourier Transform: Change of basis from time domain to frequency domain using complex exponentials
X[k] = Σn=0^{N-1} x[n] e^{-2πikn/N} for k = 0, 1, ..., N-1
where N = 4 (length of input signal), x = [1, 2, 1, 0]
X[0] = Σn=0³ x[n] e^{-2πi×0×n/4} = Σn=0³ x[n] = 1 + 2 + 1 + 0 = 4
X[1] = Σn=0³ x[n] e^{-2πi×1×n/4} = x[0]e^0 + x[1]e^{-πi/2} + x[2]e^{-πi} + x[3]e^{-3πi/2}
= 1×1 + 2×e^{-πi/2} + 1×e^{-πi} + 0×e^{-3πi/2}
= 1 + 2×(-i) + 1×(-1) + 0
= 1 - 2i - 1 = -2i
X[2] = Σn=0³ x[n] e^{-2πi×2×n/4} = Σn=0³ x[n] e^{-πin}
= x[0]e^0 + x[1]e^{-πi} + x[2]e^{-2πi} + x[3]e^{-3πi}
= 1×1 + 2×(-1) + 1×1 + 0×(-1)
= 1 - 2 + 1 + 0 = 0
X[3] = Σn=0³ x[n] e^{-2πi×3×n/4} = x[0]e^0 + x[1]e^{-3πi/2} + x[2]e^{-3πi} + x[3]e^{-9πi/2}
= 1×1 + 2×e^{-3πi/2} + 1×e^{-3πi} + 0×e^{-πi/2}
= 1 + 2×i + 1×(-1) + 0
= 1 + 2i - 1 = 2i
DFT: X = [4, -2i, 0, 2i]
Magnitude spectrum: |X| = [|4|, |-2i|, |0|, |2i|] = [4, 2, 0, 2]
Phase spectrum: arg(X) = [0, -π/2, 0, π/2]
Interpretation: - DC component (frequency 0): magnitude 4 - Fundamental frequency: magnitude 2, phase -π/2 - Second harmonic: no contribution - Third harmonic: magnitude 2, phase π/2
The signal has strong DC component and alternating pattern at fundamental frequency
• Complex exponentials: e^{iθ} = cos(θ) + i sin(θ)
• Frequency interpretation: Higher indices correspond to higher frequencies
• Hermitian symmetry: For real signals, X[N-k] = X*[k]