投影矩阵
给定列向量 b ⃗ \vec{b} b,矩阵 A \mathbf{A} A, b ⃗ \vec b b到 A A A的列空间的投影是什么?
设
A = [ a 1 ⃗ , a 2 ⃗ , ⋯ , a n ⃗ ] A= \begin{bmatrix} \vec{a_1}, \vec{a_2}, \cdots, \vec{a_n} \end{bmatrix} A=[a1,a2,⋯,an]
b ⃗ \vec b b到 A A A的列空间投影一定是 a 1 ⃗ , a 2 ⃗ , ⋯ , a n ⃗ \vec{a_1}, \vec{a_2}, \cdots, \vec{a_n} a1,a2,⋯,an的线性组合,也就是
A x ⃗ = x 1 a 1 ⃗ + x 2 a 2 ⃗ + ⋯ + x n a n ⃗ A \vec x = x_1 \vec{a_1} + x_2 \vec {a_2} + \cdots + x_n \vec{a_n} Ax=x1a1+x2a2+⋯+xnan
因此垂线段为 b ⃗ − A x ⃗ \vec b -A\vec x b−Ax
它与 a 1 ⃗ , ⋯ , a n ⃗ \vec{a_1},\cdots,\vec{a_n} a1,⋯,an张成的平面垂直,即
A T ( b ⃗ − A x ⃗ ) = 0 A^T(\vec b-A\vec x) = 0 AT(b−Ax)=0
当 A T A A^TA ATA满秩( A A A列满秩)时, A T A x ⃗ = A T b ⃗ A^TA\vec x = A^T\vec b ATAx=ATb 有唯一解
x ⃗ = ( A T A ) − 1 A T b ⃗ A x ⃗ = A ( A T A ) − 1 A T b ⃗ \vec x = (A^TA)^{-1}A^T \vec b \\ A\vec x = A(A^TA)^{-1}A^T\vec b x=(ATA)−1ATbAx=A(ATA)−1ATb
于是投影矩阵为
P = A ( A T A ) − 1 A T P = A(A^TA)^{-1}A^T P=A(ATA)−1AT
当 A T A A^TA ATA不满秩( A A A不列满秩)时, A T A x ⃗ = A T b ⃗ A^TA\vec x = A^T\vec b ATAx=ATb 有多解