7.1 Diagonalization of symmetric matrices (对称矩阵的对角化)

本文为《Linear algebra and its applications》的读书笔记

Diagonalization of symmetric matrices

A symmetric matrix is a matrix $A$ such that $A^T=A$. Such a matrix is necessarily square.

To begin the study of symmetric matrices, it is helpful to review the diagonalization process of Section 5.3.

PROOF
Let ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$ be eigenvectors that correspond to distinct eigenvalues, say, ${\lambda }_1$ and ${\lambda }_2$.

Since ${\lambda }_1\ne {\lambda }_2$, ${\mathbf{v}}_1\cdot {\mathbf{v}}_2=0$.

An $n\times n$ matrix $A$ is said to be orthogonally diagonalizable(正交对角化) if there are an orthogonal matrix $P$ (with $P^{-1}=P^T$ ) and a diagonal matrix $D$ such that

Such a diagonalization requires $n$ linearly independent and orthonormal eigenvectors.

If $A$ is orthogonally diagonalizable, then

Thus $A$ is symmetric! Theorem 2 below shows that, conversely, every symmetric matrix is orthogonally diagonalizable. The main idea for a proof will be given after Theorem 3.

This theorem is rather amazing, because the work in Chapter 5 would suggest that it is usually impossible to tell when a matrix is diagonalizable. But this is not the case for symmetric matrices.

EXAMPLE 3
Orthogonally diagonalize the matrix

, whose characteristic equation is

SOLUTION

Although ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$ are linearly independent, they are not orthogonal. Then use the projection of ${\mathbf{v}}_2$ onto ${\mathbf{v}}_1$ to produce an orthogonal set.

Then { v 1 , z 2 } \{\boldsymbol v_1,\boldsymbol z_2\} { v1​,z2​} is an orthogonal set in the eigenspace for $\lambda =7$. (Note that ${\mathbf{z}}_2$ is a linear combination of the eigenvectors ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$, so ${\mathbf{z}}_2$ is in the eigenspace.)

Normalize ${\mathbf{v}}_1$ and ${\mathbf{z}}_2$ to obtain the following orthonormal basis for the eigenspace for $\lambda =7$:

An orthonormal basis for the eigenspace for $\lambda =-2$ is

By Theorem 1, ${\mathbf{u}}_3$ is orthogonal to the other eigenvectors ${\mathbf{u}}_1$ and ${\mathbf{u}}_2$. Hence { u 1 , u 2 , u 3 } \{\boldsymbol u_1,\boldsymbol u_2,\boldsymbol u_3\} { u1​,u2​,u3​} is an orthonormal set. Let

Then $P$ orthogonally diagonalizes $A$, and $A=PD{P}^{-1}$.

The Spectral Theorem 谱定理

The set of eigenvalues of a matrix $A$ is sometimes called the $spectrum$ of $A$, and the following description of the eigenvalues is called a $spectral$ $theorem$.

• Part $(a)$ follows from Supplementary exercises in Section 5.5.
• Part $(b)$ follows easily from part (d).
• Part $(c)$ is Theorem 1.
• Because of $(a)$, a proof of $(d)$ can be found in the $Appendix$: proof of Theorem 3.

Spectral Decomposition 谱分解

Suppose $A=PD{P}^{-1}$, where the columns of $P$ are orthonormal eigenvectors ${\mathbf{u}}_1,...,{\mathbf{u}}_n$ of $A$ and the corresponding eigenvalues ${\lambda }_1,...,{\lambda }_n$ are in the diagonal matrix $D$. Then, since $P^{-1}=P^T$ ,

This representation of $A$ is called a spectral decomposition of $A$ because it breaks up $A$ into pieces determined by the spectrum (eigenvalues) of $A$.

• Each term in (2) is an $n\times n$ matrix of rank 1. For example, every column of ${\lambda }_1{\mathbf{u}}_1{\mathbf{u}}_1^T$ is a multiple of ${\mathbf{u}}_1$.
• Furthermore, each matrix ${\mathbf{u}}_j{\mathbf{u}}_j^T$ is a projection matrix(投影矩阵) in the sense that for each $\mathbf{x}$ in ${\mathbb{R}}^n$, the vector $({\mathbf{u}}_j{\mathbf{u}}_j^T)\mathbf{x}$ is the orthogonal projection of $\mathbf{x}$ onto the subspace spanned by ${\mathbf{u}}_j$ .
  PROOF
  $({\mathbf{u}}_j{\mathbf{u}}_j^T)\mathbf{x}={\mathbf{u}}_j({\mathbf{u}}_j^T\mathbf{x})=({\mathbf{u}}_j^T\mathbf{x}){\mathbf{u}}_j=({\mathbf{u}}_j\cdot \mathbf{x}){\mathbf{u}}_j$. (${\mathbf{u}}_j^T\mathbf{x}$ is a scaler) This is the orthogonal projection of $\mathbf{x}$ onto $\mathbf{u}$.
  

EXERCISE
Let $A$ be an $n\times n$ symmetric matrix of rank $r$. Explain why the spectral decomposition of $A$ represents $A$ as the sum of $r$ rank 1 matrices.
SOLUTION
[Hint: $dimNulA=n-r$]

Appendix: proof of Theorem 3 (d)

The Schur factorization(舒尔分解) of an $n\times n$ matrix $A$ is in the form $A=UR{U}^T$ , where $U$ is an orthogonal matrix and $R$ is an $n\times n$ upper triangular matrix.

THEOREM
Let $A$ be an $n\times n$ matrix with $n$ real eigenvalues, counting multiplicities, denoted by ${\lambda }_1,...,{\lambda }_n$. It can be shown that $A$ admits a (real) Schur factorization.
PROOF
Parts (a) and (b) show the key ideas in the proof. The rest of the proof amounts to repeating (a) and (b) for successively smaller matrices, and then piecing together the results.
a. Let ${\mathbf{u}}_1$ be a unit eigenvector corresponding to ${\lambda }_1$, let ${\mathbf{u}}_2,...,{\mathbf{u}}_n$ be any other vectors such that { u 1 , . . . , u n } \{\boldsymbol u_1,...,\boldsymbol u_n\} { u1​,...,un​} is an orthonormal basis for ${\mathbb{R}}^n$, and then let $U=\left[\begin{array}{cccc}{\mathbf{u}}_1& {\mathbf{u}}_2& ...& {\mathbf{u}}_n\end{array}\right]$. It can be shown that the first column of $U^{T}AU$ is ${\lambda }_1{\mathbf{e}}_1$, where ${\mathbf{e}}_1$ is the first column of the $n\times n$ identity matrix.
b. Part (a) implies that $U^{T}AU$ has the form shown below.

Since $det(U^{T}AU-\lambda I)=det(U^{T}AU-\lambda U^{T}U)=det(U^T(A-\lambda I)U)=det(U^T)det(A-\lambda I)det(U)=det(U^{-1})det(A-\lambda I)det(U)=det(A-\lambda I)$, the characteristic polynomials of $U^{T}AU$ and $A$ are the same. Thus $U^{T}AU$ and $A$ have the same eigenvalues, which indicates that the eigenvalues of $A_1$ are ${\lambda }_2,...,{\lambda }_n$.

Similar to (a), let ${\mathbf{u}}_2^{\prime }$ be a unit eigenvector corresponding to ${\lambda }_2$, let ${\mathbf{u}}_3^{\prime },...,{\mathbf{u}}_n^{\prime }$ be any other vectors such that { u 2 ′ , . . . , u n ′ } \{\boldsymbol u_2',...,\boldsymbol u_n'\} { u2′​,...,un′​} is an orthonormal basis for ${\mathbb{R}}^{n-1}$, and then let $U^{\prime }=\left[\begin{array}{cccc}{\mathbf{u}}_2^{\prime }& {\mathbf{u}}_3^{\prime }& ...& {\mathbf{u}}_n^{\prime }\end{array}\right]$. It can be shown that the first column of $U^{\prime T}{A}_1{U}^{\prime }$ is ${\lambda }_2{\mathbf{e}}_1^{\prime }$, where ${\mathbf{e}}_1^{\prime }$ is the first column of the $(n-1)\times (n-1)$ identity matrix. So $U^{\prime T}{A}_1{U}^{\prime }$ has the form similar to $U^{T}AU$.

Suppose $U^{T}AU=\left[\begin{array}{cc}{\lambda }_1& {\mathbf{x}}^T\\ 0& A_1\end{array}\right]$, then
$$\begin{array}{rl}\left[\begin{array}{cc}1& 0\\ 0& U_1^T\end{array}\right]U^{T}AU\left[\begin{array}{cc}1& 0\\ 0& U_1\end{array}\right]& =\left[\begin{array}{cc}1& 0\\ 0& U_1^T\end{array}\right]\left[\begin{array}{cc}{\lambda }_1& {\mathbf{x}}^T\\ 0& A_1\end{array}\right]\left[\begin{array}{cc}1& 0\\ 0& U_1\end{array}\right]\\ & =\left[\begin{array}{cc}{\lambda }_1& {\mathbf{x}}^T{U}_1\\ 0& U_1^T{A}_1{U}_1\end{array}\right]\end{array}$$

Let $U^{\prime }=U\left[\begin{array}{cc}1& 0\\ 0& U_1\end{array}\right]$, it can be shown that $U^{\prime }$ is an orthogonal matrix. So
$$U^{\prime T}A{U}^{\prime }=\left[\begin{array}{ccccc}{\lambda }_1& \ast & \ast & \ast & \ast \\ 0& {\lambda }_2& \ast & \ast & \ast \\ ...& 0& & & \\ ...& ...& & A_2& \\ 0& 0& & & \end{array}\right]$$

Continue this process and we will finally get a (real) Schur factorization of $A$.

---

With the theorem above, the proof is quite easy.

Let $A$ be a symmetric matrix. Since $A$ has $n$ real eigenvalues, counting multiplicities, $A$ has a real Schur factorization $UR{U}^T$. Since $A^T=U{R}^T{U}^T=A=UR{U}^T$, $R=R^T$, which indicates that $R$ is in fact a diagonal matrix, with eigenvalues on its main diagonal.

Thus $A=UR{U}^{-1}$, where $U$ is an orthogonal matrix and $R$ is a diagonal matrix. So $A$ is orthogonally diaganalizable.