Learning source: "Matrix Analysis and Application" Zhang Xianda Tsinghua University Press

singular value decomposition

1. The relationship between matrix singular value and matrix

The singular value of a matrix is closely related to the norm, determinant, condition number, and eigenvalue of the matrix.

1. The relationship between singular value and norm

$A$ The spectral norm of the matrix is equal to $A$ the largest singular value of

$\left \| A \right \|_{spec}=\sigma_1$

According to the singular value decomposition theorem of the matrix, since $A$ the Frobenius norm of the matrix $\left \| A \right \|_F$ is unitary invariant, that is

$\left \| U^HAV \right \|_F=\left \| A \right \|_F$

Therefore, there are

$\left \| A \right \|_F=\left [ \sum_{i=1}^{m}\sum_{j=1}^{n}\left | a_{ij} \right |^2 \right ]^{\frac{1}{2}}$

$=\left \| U^HAV \right \|_F-\left \| \Sigma \right \|_F$

$=\sqrt{\sigma_1^2+\sigma_2^2+\cdots +\sigma_r^2}$

That is, the Frobenius norm of any matrix is equal to the positive square root of the sum of the squares of all nonzero singular values of that matrix.

2. The relationship between singular value and determinant

Let $A$ be $n\times n$ a square matrix. Since the absolute value of the determinant of the unitary matrix is 1, there is

$\left | det(A) \right |=\left | det\Sigma \right |=\sigma_1\sigma_2\cdots \sigma_n$

When all $\sigma_i$ is non-zero, it $A$ is non-singular; when existence $\sigma_i$ is zero, it $A$ is singular.

For a $n\times n$ matrix $A$ , the following inequalities hold:

$\left\{\begin{matrix} n\sigma_1\geqslant \left \| A \right \|_F \geqslant \sigma_1\\ \sigma_1^n\geqslant \sigma_1^{n-1}\sigma_n\geqslant \left | det(A) \right |\geqslant \sigma_n^n\\ \left \| A \right \|_F\geqslant \sigma_1\geqslant \left | det(A) \right |^{\frac{1}{n}}\\ \left | det(A) \right |^{\frac{1}{n}}\geqslant \sigma_n\geqslant \frac{\left | det(A) \right |}{\left \| A \right \|_F^{n-1}}\\ \frac{\left \| A \right \|_F^n}{\left | det(A) \right |}\geqslant \frac{\sigma_1}{\sigma_n}\geqslant max\left \{ 1,\frac{1}{n}\frac{\left \| A \right \|_F}{\left | det(A) \right |^{\frac{1}{n}}} \right \} \end{matrix}\right.$

3. The relationship between singular value and condition number

For a $m\times n$ matrix $A$ , its condition number can be defined as

$cond(A)=\frac{\sigma_1}{\sigma_p},\qquad p=min\left \{ m,n \right \}]$

Because $\sigma_1 \geqslant \sigma_p$ , the condition number is a positive number greater than or equal to 1. For a singular matrix, since there is at least one singular value $\sigma_p=0$ , the condition number of the singular matrix is infinite; when $A$ the condition number of a matrix is not infinite but very large, the matrix $A$ is said to be close to singular. It means that when $A$ the condition number of the matrix is large, $A$ the linear dependence of the row vector or column vector of the matrix is strong.

For the equation $Ax=b$ , $A^HA$ the singular value decomposition of

$A^HA=V\Sigma^2V^H$

$A^HA$ That is, the largest and smallest singular values of the matrix are $A$ the squares of the largest and smallest singular values of the matrix, respectively, so

$cond(A^HA)=\frac{\sigma_1^2}{\sigma_n^2}=\left [ cond(A) \right ]^2$

That is, $A^HA$ the condition number of the matrix is $A$ the square of the condition number of the matrix.

4. The relationship between singular value and eigenvalue

Let the eigenvalues and singular values of $n\times n$ the square symmetric matrix be $A$ $\lambda_1,\lambda_2,\cdots ,\lambda_n(\left | \lambda_1 \right | \geqslant \left | \lambda_2 \right | \geqslant \cdots \geqslant \left | \lambda_n \right |)$ $\sigma_1,\sigma_2,\cdots ,\sigma_n(\sigma_1\geqslant \sigma_2\geqslant \cdots \geqslant \sigma_n\geqslant 0)$

$\sigma_1\geqslant \left | \lambda_i \right |\geqslant \sigma_n(i=1,2,\cdots ,n),\quad cond(A)\geqslant \frac{\left | \lambda_1 \right |}{\left | \lambda_n \right |}$

2. Summary of properties of singular values

1. The equation relationship that singular values obey

1) A matrix $A_{m\times n}$ and its complex conjugate transpose $A^H$ have the same singular values.

2) $A_{m\times n}$ The nonzero singular values of the matrix are the positive square roots of the nonzero eigenvalues of $AA^H$ or . $A^HA$

3) $\sigma> 0$ is $A_{m\times n}$ a single singular value of matrix-matrix if and only if $\sigma^2$ it is a single eigenvalue of $AA^H$ or . $A^HA$

4) If $p=min\left \{ m,n \right \}$ and $\sigma_1,\sigma_2,\cdots ,\sigma_p$ is $A_{m\times n}$ a singular value of the matrix, then

$tr(A^HA)=\sum_{i=1}^{p}\sigma_i^2$

5) The absolute value of the matrix determinant is equal to the product of the singular values of the matrix, namely

$\left | det(A) \right |=\sigma_1\sigma_2\cdots \sigma_n$

6) $A$ The spectral norm of the matrix is equal to $A$ the largest singular value of , ie $\left \| A \right \|_{spec}=\sigma_{max}$ .

7) If $m\geqslant n$ , then for the matrix $A_{m\times n}$ , there is

$\sigma_{min}(A)=min\left \{ \left $\frac{x^HA^HAx}{x^Hx}\right $^{\frac{1}{2}}:x\neq 0 \right \}$

$=min\left \{ \left $x^HA^HAx \right $^{\frac{1}{2}} :x^Hx=1,x\in C^n\right \}$

8) If $m\geqslant n$ , then for the matrix $A_{m\times n}$ , there is

$\sigma_{max}(A)=max\left \{ \left $\frac{x^HA^HAx}{x^Hx}\right $^{\frac{1}{2}}:x\neq 0 \right \}$

$=max\left \{ \left $x^HA^HAx \right $^{\frac{1}{2}} :x^Hx=1,x\in C^n\right \}$

9) If $m\times m$ the matrix $A$ is non-singular, then

$\frac{1}{\sigma_{min}(A)}=max\left \{ \left ( \frac{x^H(A^{-1})^HA^{-1}x}{x^Hx} \right )^{\frac{1}{2}}:x\neq 0,x\in C^n \right \}$

10) $A=U\begin{bmatrix} \Sigma_1 &O \\ O& O \end{bmatrix}V^H$ If the singular value decomposition of $m\times n$ the matrix , then the Moose-Penrose inverse matrix $A$ $A$

$A^\dagger=V\begin{bmatrix} \Sigma_1^{-1} &O \\ O& O \end{bmatrix}U^H$

11) $\sigma_1,\sigma_2,\cdots ,\sigma_p$ If $m\times n$ the matrix $A$ has non-zero singular values (where, $p=min\left \{ m,n \right \}$ ), then the matrix $\begin{bmatrix} O & A\\ A^H& O \end{bmatrix}$ has $2p$ non-zero singular values $\sigma_1,\cdots ,\sigma_p,-\sigma_1,\cdots ,-\sigma_p$ and $\left | m-n \right |$ zero singular values.

2. Inequality relation of singular value

1) If $A$ the sum $B$ is $m\times n$ a matrix, then for $1\leqslant i,j\leqslant p,i+j\leqslant p+1(p=min\left \{ m,n \right \})$ , have

$\sigma_{i+j-1}(A+B)\leqslant \sigma_i(A)+\sigma_j(B)$

In particular, $j=1$ at the time , $\sigma_i(A+B)\leqslant \sigma_i(A)+\sigma_i(B),i=1,2,\cdots ,p$ established.

2) For the matrix $A_{m\times n},B_{m\times n}$ , there is

$\sigma_{max}(A+B)\leqslant \sigma_{max}(A)+\sigma_{max}(B)$

3) If $A$ the sum $B$ is $m\times n$ a matrix, then

$\sum_{j=1}^{p}\left [ \sigma_j(A+B)-\sigma_j(A) \right ]^2 \leqslant \sigma_{max}(A)+\sigma_{max}(B)$

4) If $A_{m\times n}=[a_1,a_2,\cdots ,a_m]$ the singular value of $\sigma_1(A)\geqslant \sigma_2(A)\geqslant \cdots \geqslant \sigma_m(A)$ , then

$\sum_{j=1}^{k}[\sigma_{m-k+j}(A)]^2\leqslant \sum_{j=1}^{k}a_j^Ha_j\leqslant \sum_{j=1}^{k}[\sigma_j(A)]^2,\quad k=1,2,\cdots ,m$

5) If $p=min\left \{ m,n \right \}$ , and the singular values of and are arranged as $A_{m\times n}$ , and , then $B_{m\times n}$ $\sigma_1(A)\geqslant \sigma_2(A)\geqslant \cdots \geqslant \sigma_p(A)$ $\sigma_1(B)\geqslant \sigma_2(B)\geqslant \cdots \geqslant \sigma_p(B)$ $\sigma_1(A+B)\geqslant \sigma_2(A+B)\geqslant \cdots \geqslant \sigma_p(A+B)$

$\sigma_{i+j-1}(AB^H)\leqslant \sigma_i(A)\sigma_j(B),\quad 1\leqslant i,j\leqslant p,\quad i+j\leqslant p+1$

6) Suppose $m\times (n-1)$ the matrix is a matrix obtained by $B$ removing any column of $m\times n$ the matrix , and their singular values are arranged in non-descending order, then $A$

$\sigma_1(A)\geqslant \sigma_1(B)\geqslant \sigma_2(A)\geqslant \sigma_2(B)\geqslant \cdots \geqslant \sigma_h(A)\geqslant \sigma_h(B)\geqslant 0$

In the formula, $h=min\left \{ m,n-1 \right \}$ .

7) Suppose $(m-1)\times n$ the matrix $B$ is a matrix obtained by removing any row of $m\times n$ the matrix $A$ , and their singular values are arranged in non-descending order, then

$\sigma_1(A)\geqslant \sigma_1(B)\geqslant \sigma_2(A)\geqslant \sigma_2(B)\geqslant \cdots \geqslant \sigma_h(A)\geqslant \sigma_h(B)\geqslant 0$

In the formula, $h=min\left \{ m,n-1 \right \}$ .

8) $A_{m\times n}$ The largest singular value of the matrix satisfies the inequality

$\sigma_{max }(A)\geqslant \left [ \frac{1}{n}tr(A^HA) \right ]^{\frac{1}{2}}$

Matrix Analysis and Application(20)

singular value decomposition

1. The relationship between matrix singular value and matrix

1. The relationship between singular value and norm

2. The relationship between singular value and determinant

3. The relationship between singular value and condition number

4. The relationship between singular value and eigenvalue

2. Summary of properties of singular values

1. The equation relationship that singular values obey

2. Inequality relation of singular value

Guess you like

Matrix Analysis and Application(20)

singular value decomposition

1. The relationship between matrix singular value and matrix

1. The relationship between singular value and norm

2. The relationship between singular value and determinant

3. The relationship between singular value and condition number

4. The relationship between singular value and eigenvalue

2. Summary of properties of singular values

1. The equation relationship that singular values ​​obey

2. Inequality relation of singular value

Guess you like

1. The equation relationship that singular values obey