Matrix Multiplication - Comprehensive Summary
Matrix Multiplication: For matrices A (m×n) and B (n×p), the product AB is an (m×p) matrix where \((AB)_{ij} = \sum_{k=1}^{n} A_{ik}B_{kj}\).
Compatibility: Matrix multiplication AB is defined only when the number of columns of A equals the number of rows of B.
Dot Product Interpretation: Entry (i,j) of AB is the dot product of row i of A with column j of B.
• Not commutative: AB ≠ BA in general
• Associative: (AB)C = A(BC)
• Distributive: A(B+C) = AB + AC
• Identity: AI = IA = A (when dimensions match)
1. Row-by-column: Compute dot product of each row of A with each column of B
2. Column interpretation: Each column of AB is A times the corresponding column of B
3. Row interpretation: Each row of AB is the corresponding row of A times B
4. Linear combination: Each column of AB is linear combination of columns of A
• Always check dimensions before multiplying
• Use the "inner-outer" rule: (m×n)(n×p) → (m×p)
• Practice with small matrices to build intuition
• Remember that AB may exist while BA does not
Matrix Multiplication: For matrices A (m×n) and B (n×p), the entry in row i and column j of the product AB is the dot product of row i of A with column j of B: \((AB)_{ij} = \sum_{k=1}^{n} A_{ik}B_{kj}\).
- Verify compatibility: number of columns of A equals number of rows of B
- Compute each entry of the result matrix using dot products
- For entry (i,j): multiply row i of A by column j of B
- Arrange results in the correct positions
Matrix A is 2×2 and matrix B is 2×2. The number of columns of A (2) equals the number of rows of B (2), so multiplication is possible. The result will be a 2×2 matrix.
\((AB)_{11} = \text{row 1 of A} \cdot \text{column 1 of B} = (1,2) \cdot (5,7) = (1)(5) + (2)(7) = 5 + 14 = 19\)
\((AB)_{12} = \text{row 1 of A} \cdot \text{column 2 of B} = (1,2) \cdot (6,8) = (1)(6) + (2)(8) = 6 + 16 = 22\)
\((AB)_{21} = \text{row 2 of A} \cdot \text{column 1 of B} = (3,4) \cdot (5,7) = (3)(5) + (4)(7) = 15 + 28 = 43\)
\((AB)_{22} = \text{row 2 of A} \cdot \text{column 2 of B} = (3,4) \cdot (6,8) = (3)(6) + (4)(8) = 18 + 32 = 50\)
\(AB = \begin{pmatrix} 19 & 22 \\ 43 & 50 \end{pmatrix}\)
The product of A and B is \(\begin{pmatrix} 19 & 22 \\ 43 & 50 \end{pmatrix}\).
• Dimension compatibility: Check that inner dimensions match
• Dot product calculation: Multiply corresponding elements and sum
• Position correspondence: Row i of A with column j of B gives entry (i,j)
Rectangular Matrix Multiplication: For C (m×n) and D (n×p), CD exists and results in an (m×p) matrix. The product DC exists only if p = m.
C is 2×3 and D is 3×2, so CD is 2×2.
\((CD)_{11} = (1,2,3) \cdot (7,9,11) = 1(7) + 2(9) + 3(11) = 7 + 18 + 33 = 58\)
\((CD)_{12} = (1,2,3) \cdot (8,10,12) = 1(8) + 2(10) + 3(12) = 8 + 20 + 36 = 64\)
\((CD)_{21} = (4,5,6) \cdot (7,9,11) = 4(7) + 5(9) + 6(11) = 28 + 45 + 66 = 139\)
\((CD)_{22} = (4,5,6) \cdot (8,10,12) = 4(8) + 5(10) + 6(12) = 32 + 50 + 72 = 154\)
So: \(CD = \begin{pmatrix} 58 & 64 \\ 139 & 154 \end{pmatrix}\)
D is 3×2 and C is 2×3, so DC is 3×3.
\((DC)_{11} = (7,8) \cdot (1,4) = 7(1) + 8(4) = 7 + 32 = 39\)
\((DC)_{12} = (7,8) \cdot (2,5) = 7(2) + 8(5) = 14 + 40 = 54\)
\((DC)_{13} = (7,8) \cdot (3,6) = 7(3) + 8(6) = 21 + 48 = 69\)
\((DC)_{21} = (9,10) \cdot (1,4) = 9(1) + 10(4) = 9 + 40 = 49\)
\((DC)_{22} = (9,10) \cdot (2,5) = 9(2) + 10(5) = 18 + 50 = 68\)
\((DC)_{23} = (9,10) \cdot (3,6) = 9(3) + 10(6) = 27 + 60 = 87\)
\((DC)_{31} = (11,12) \cdot (1,4) = 11(1) + 12(4) = 11 + 48 = 59\)
\((DC)_{32} = (11,12) \cdot (2,5) = 11(2) + 12(5) = 22 + 60 = 82\)
\((DC)_{33} = (11,12) \cdot (3,6) = 11(3) + 12(6) = 33 + 72 = 105\)
So: \(DC = \begin{pmatrix} 39 & 54 & 69 \\ 49 & 68 & 87 \\ 59 & 82 & 105 \end{pmatrix}\)
CD is 2×2 while DC is 3×3, so they are different matrices entirely.
This demonstrates that matrix multiplication is not commutative.
\(CD = \begin{pmatrix} 58 & 64 \\ 139 & 154 \end{pmatrix}\) and \(DC = \begin{pmatrix} 39 & 54 & 69 \\ 49 & 68 & 87 \\ 59 & 82 & 105 \end{pmatrix}\). Both products exist but are different matrices.
• Non-commutativity: AB ≠ BA in general, even when both exist
• Dimension preservation: Result dimensions depend on outer dimensions
• Compatibility check: Inner dimensions must match for multiplication to exist
Matrix-Vector Product: A special case of matrix multiplication where the second matrix is a column vector. The result is a column vector where each entry is the dot product of the corresponding row of M with the vector x.
Matrix M is 3×3 and vector x is 3×1, so the product Mx is 3×1 (a 3-dimensional vector).
\((M\vec{x})_1 = \text{row 1 of M} \cdot \vec{x} = (2,1,0) \cdot (1,2,3) = 2(1) + 1(2) + 0(3) = 2 + 2 + 0 = 4\)
\((M\vec{x})_2 = \text{row 2 of M} \cdot \vec{x} = (-1,3,1) \cdot (1,2,3) = (-1)(1) + 3(2) + 1(3) = -1 + 6 + 3 = 8\)
\((M\vec{x})_3 = \text{row 3 of M} \cdot \vec{x} = (0,2,-1) \cdot (1,2,3) = 0(1) + 2(2) + (-1)(3) = 0 + 4 - 3 = 1\)
\(M\vec{x} = \begin{pmatrix} 4 \\ 8 \\ 1 \end{pmatrix}\)
We can also view this as a linear combination of the columns of M:
\(M\vec{x} = 1\begin{pmatrix} 2 \\ -1 \\ 0 \end{pmatrix} + 2\begin{pmatrix} 1 \\ 3 \\ 2 \end{pmatrix} + 3\begin{pmatrix} 0 \\ 1 \\ -1 \end{pmatrix} = \begin{pmatrix} 2 \\ -1 \\ 0 \end{pmatrix} + \begin{pmatrix} 2 \\ 6 \\ 4 \end{pmatrix} + \begin{pmatrix} 0 \\ 3 \\ -3 \end{pmatrix} = \begin{pmatrix} 4 \\ 8 \\ 1 \end{pmatrix}\)
The product \(M\vec{x} = \begin{pmatrix} 4 \\ 8 \\ 1 \end{pmatrix}\).
• Row-vector dot product: Each row of M dotted with vector x
• Linear combination: Result is weighted sum of matrix columns
• Dimension consistency: Result has same rows as M
Standard Matrix: For a linear transformation \(T: \mathbb{R}^n \to \mathbb{R}^m\), the standard matrix A is formed by applying T to the standard basis vectors: \(A = [T(\vec{e_1}) \, T(\vec{e_2}) \, \ldots \, T(\vec{e_n})]\).
Check additivity: \(T\left(\begin{pmatrix} x_1 \\ y_1 \end{pmatrix} + \begin{pmatrix} x_2 \\ y_2 \end{pmatrix}\right) = T\begin{pmatrix} x_1+x_2 \\ y_1+y_2 \end{pmatrix} = \begin{pmatrix} 2(x_1+x_2) + 3(y_1+y_2) \\ (x_1+x_2) - (y_1+y_2) \end{pmatrix} = \begin{pmatrix} 2x_1 + 3y_1 \\ x_1 - y_1 \end{pmatrix} + \begin{pmatrix} 2x_2 + 3y_2 \\ x_2 - y_2 \end{pmatrix} = T\begin{pmatrix} x_1 \\ y_1 \end{pmatrix} + T\begin{pmatrix} x_2 \\ y_2 \end{pmatrix}\). ✓
\(T\begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 2(1) + 3(0) \\ 1 - 0 \end{pmatrix} = \begin{pmatrix} 2 \\ 1 \end{pmatrix}\)
\(T\begin{pmatrix} 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 2(0) + 3(1) \\ 0 - 1 \end{pmatrix} = \begin{pmatrix} 3 \\ -1 \end{pmatrix}\)
The standard matrix A has columns \(T(\vec{e_1})\) and \(T(\vec{e_2})\):
\(A = \begin{pmatrix} 2 & 3 \\ 1 & -1 \end{pmatrix}\)
Check: \(A\begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 2 & 3 \\ 1 & -1 \end{pmatrix}\begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 2x + 3y \\ x - y \end{pmatrix} = T\begin{pmatrix} x \\ y \end{pmatrix}\). ✓
The standard matrix is \(A = \begin{pmatrix} 2 & 3 \\ 1 & -1 \end{pmatrix}\).
• Standard matrix theorem: Every linear transformation has a unique matrix representation
• Column method: Standard matrix columns are transformations of basis vectors
• Verification: Always check that T(x) = Ax for arbitrary x
Composition of Transformations: \((T \circ S)(\vec{x}) = T(S(\vec{x}))\). The standard matrix of the composition is the product of the individual standard matrices: \(A_{T \circ S} = A_T A_S\).
The composition \((T \circ S)(\vec{x}) = T(S(\vec{x}))\) means we apply S first, then T. The standard matrix for the composition is the product of the individual matrices in the order T · S.
\(A_{T \circ S} = A_T A_S = \begin{pmatrix} 2 & 0 \\ 1 & 1 \end{pmatrix}\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}\)
\((A_{T \circ S})_{11} = (2,0) \cdot (1,3) = 2(1) + 0(3) = 2\)
\((A_{T \circ S})_{12} = (2,0) \cdot (2,4) = 2(2) + 0(4) = 4\)
\((A_{T \circ S})_{21} = (1,1) \cdot (1,3) = 1(1) + 1(3) = 4\)
\((A_{T \circ S})_{22} = (1,1) \cdot (2,4) = 1(2) + 1(4) = 6\)
\(A_{T \circ S} = \begin{pmatrix} 2 & 4 \\ 4 & 6 \end{pmatrix}\)
Let \(\vec{x} = \begin{pmatrix} 1 \\ 0 \end{pmatrix}\).
\(S(\vec{x}) = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}\begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 1 \\ 3 \end{pmatrix}\)
\(T(S(\vec{x})) = \begin{pmatrix} 2 & 0 \\ 1 & 1 \end{pmatrix}\begin{pmatrix} 1 \\ 3 \end{pmatrix} = \begin{pmatrix} 2 \\ 4 \end{pmatrix}\)
\((T \circ S)(\vec{x}) = \begin{pmatrix} 2 & 4 \\ 4 & 6 \end{pmatrix}\begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 2 \\ 4 \end{pmatrix}\). ✓
The standard matrix for \(T \circ S\) is \(\begin{pmatrix} 2 & 4 \\ 4 & 6 \end{pmatrix}\).
• Composition property: Standard matrix of composition is product of individual matrices
• Order matters: \((T \circ S)(\vec{x})\) corresponds to \(A_T A_S\), not \(A_S A_T\)
• Verification: Check with specific vector to confirm correctness
Matrix Multiplication: A binary operation that produces a matrix from two matrices. The resulting matrix has entries that are dot products of rows and columns.
Linear Transformation: A function that preserves vector addition and scalar multiplication, always representable as matrix multiplication.
Standard Matrix: The unique matrix that represents a linear transformation with respect to standard bases.
- Before multiplying: Check that the number of columns of the first matrix equals the number of rows of the second
- For each entry: Compute the dot product of the corresponding row and column
- For verification: Use alternative interpretations (linear combinations of columns/rows)
- For compositions: Multiply matrices in the order of transformation application
• Associativity: \((AB)C = A(BC)\)
• Distributivity: \(A(B+C) = AB + AC\) and \((A+B)C = AC + BC\)
• Identity: \(AI = IA = A\) where I is the identity matrix
• Zero matrix: \(A0 = 0A = 0\)
• \((AB)^T = B^T A^T\) (transpose property)
Geometric Interpretation: Matrix multiplication corresponds to applying linear transformations sequentially. The product matrix represents the combined effect of both transformations.
- Original vector: \(\vec{v} = \begin{pmatrix} 1 \\ 0 \end{pmatrix}\)
- After transformation A: \(\vec{v'} = A\vec{v}\)
- After transformation B: \(\vec{v''} = B(A\vec{v}) = (BA)\vec{v}\)
- Direct transformation: \(\vec{v''} = (BA)\vec{v}\)