Solving Overdetermined Equation Ax=0 and Singular Value Decomposition

Reference:

1. Solve the overdetermined equation Ax=0

Article jump:

1. Singular value decomposition SVD and principal component analysis PCA
2. Lagrange multiplier method

The singular value decomposition has been introduced in the previous article, now let's talk about an application of the singular value decomposition.

Many problems in engineering will come down to solving the overdetermined equation $\mathbf{Ax}=0$，$\mathbf{A}$ 是 $m\times A\; matrix\; of\; n$ , and$m>n$ . For example , triangulated map pointsin SLAM,PnPand other problems are all solving this equation.
Obviously, this equation has a zero solution, but that's not what we want, we actually want a non-zero solution.

In order to find a non-zero solution, we $x$ plus a constraint$\Vert \mathbf{x}{\Vert }^2=1$ . That is to limit$x$ has length 1. and constructed as a constrained least squares problem:
$$\hat{\mathbf{x}}=\arg \min \Vert \mathbf{Ax}{\Vert }^2,\text{ subject to }\Vert \mathbf{x}{\Vert }^2=1\text{\hspace{1em}( 1 )}$$ This is a least squares problem with constraints, we move outLagrangian:L
$$\begin{array}{rl}L(\mathbf{x},l)& =\Vert \mathbf{Ax}{\Vert }^2+l\left(1-\Vert \mathbf{x}{\Vert }^2\right)\\ & ={\mathbf{x}}^T{A}^T\mathbf{Ax}+l\left(1-x^T\mathbf{x}\right)\end{array}\text{\hspace{1em}( 2 )}$$ In order to find the extremum, we respectively for$x$和λ \lambdaCalculate the partial derivative of$\lambda$$0$ :
$$\begin{array}{l}\frac{\partial L(x,\lambda }{\partial \mathbf{x}}=2A{\_}^{T}Ax-2lx=0\end{array}\text{\hspace{1em}( 3 )}$$$$\begin{array}{l}\frac{\partial L(x,\lambda }{\partial \lambda }=1-{\mathbf{x}}^T\mathbf{x}=0\end{array}\text{\hspace{1em}( 4 )}$$ Let ( 3 ) be a function of:
$$\begin{array}{r}({\mathbf{A}}^{T}A-I)\_x=0\end{array}\text{\hspace{1em}( 5 )}$$$$\begin{array}{r}{\mathbf{A}}^T\mathbf{Ax}=\lambda \mathbf{x}\end{array}\text{\hspace{1em}( 6 )}$$ It can be seen that$\lambda$ 和 $x$ are ATA \mathbf{A}^T \mathbf{A}respectively${\mathbf{A}}^{}$The特征值sumof${}^T$$A.$特征向量That is to say, the solution of formula (1) is one of these eigenvectors.
The question is, so many eigenvectors, which one should be chosen as the solution? We expand$\Vert \mathbf{Ax}{\Vert }^{}$Exercise${}^2$
$$\Vert Ax{\Vert }^2={\mathbf{x}}^T{A}^{T}Ax=x^T\lambda \mathbf{x}=\lambda {\mathbf{x}}^T\mathbf{x}=l\text{\hspace{1em}(7)}$$

The derivation of formula (7) above uses formula (6) and $\Vert \mathbf{x}{\Vert }^2=1$。

That is, we want $\Vert \mathbf{Ax}{\Vert }^2$ is the smallest, we need$\lambda$ is the smallest.
Thenthe non-zero solution of equation (1) is${\mathbf{A}}^{T}A$minimum eigenvalueλ \lambdaThe eigenvector corresponding to$\lambda$ is the right singular vector corresponding to the smallest singular value .