When deriving and using SVD to decompose and solve equations, the norm-preserving property of orthogonal matrix is used.
1. Orthogonal matrix definition
A ⊺ \mathbf{A}^\intercal A⊺A=A A ⊺ \mathbf{A}^\intercal A⊺ = E
2. Norm preservation of orthogonal matrix
The orthogonal matrix performs orthogonal transformation on the vector, and the orthogonal transformation does not change the length of the vector (norm):
suppose the orthogonal transformation of X is AX, then the norm of AX is:
It can be seen that the norm of AX is equal to the norm of X.
3. Application of SVD in solving equations
SVD decomposition of A: A=UDVT (T stands for transpose)
where U and V are orthogonal matrices.
(1) When using singular values to solve the overdetermined equation Ax=b, it will be used to find the minimum value of ||Ax-b||, ||Ax-b||=||UDVTx-b||
Next, ||UDVT-b||=||DVTx-UTb||
The above formula is to apply the norm-preserving property of the orthogonal transformation to
multiply the vector UDVT-b by the orthogonal matrix UT.
(2) Under the constraint condition ||x||=1, find the x that minimizes ||Ax||.
Suppose A=UDVT, then the problem becomes to find the minimum value of ||UDVTx||.
Norm preservation by the orthogonal matrix:
||UDVTx||=||DVTx|| ,||x|| = ||VTx||
The problem becomes to find the minimum value of ||DVTx|| under the constraint condition IIVTxll=1.
Let y=VTx, then the problem is simplified to the
constraint condition ||y||=1, find the minimum value of ||Dy||