6.2 Orthogonal sets (正交基)

本文为《Linear algebra and its applications》的读书笔记

Orthogonal sets

A set of vectors { u 1 , . . . , u p } \{\boldsymbol u_1,..., \boldsymbol u_p\} { u1​,...,up​} in ${\mathbb{R}}^n$ is said to be an orthogonal set if each pair of distinct vectors from the set is orthogonal, that is, if ${\mathbf{u}}_i\cdot {\mathbf{u}}_j=0$ whenever $i\ne j$ .

PROOF
If $0=c_1{\mathbf{u}}_1+...+c_p{\mathbf{u}}_p$ for some scalars $c_1,...,c_p$, then

because ${\mathbf{u}}_1$ is orthogonal to ${\mathbf{u}}_2,...,{\mathbf{u}}_p$. Since ${\mathbf{u}}_1$ is nonzero, ${\mathbf{u}}_1\cdot {\mathbf{u}}_1$ is not zero and so $c_1=0$. Similarly, $c_2,...,c_p$ must be zero. Thus $S$ is linearly independent.

The next theorem suggests why an orthogonal basis is much nicer than other bases. The weights in a linear combination can be computed easily.

An Orthogonal Projection 正交投影

Given a nonzero vector $\mathbf{u}$ in ${\mathbb{R}}^n$, consider the problem of decomposing a vector $\mathbf{y}$ in ${\mathbb{R}}^n$ into the sum of two vectors, one a multiple of $\mathbf{u}$ and the other orthogonal to $\mathbf{u}$. We wish to write
$$\mathbf{y}=\hat{\mathbf{y}}+\mathbf{z}(1)$$

where $\hat{\mathbf{y}}=\alpha \mathbf{u}$ for some scalar $\alpha$ and $\mathbf{z}$ is some vector orthogonal to $\mathbf{u}$. See Figure 2.

Given any scalar $\alpha$, let $\mathbf{z}=\mathbf{y}-\alpha \mathbf{u}$, so that (1) is satisfied. Then $\mathbf{y}-\hat{\mathbf{y}}$ is orthogonal to $\mathbf{u}$ if and only if

$$0=(\mathbf{y}-\alpha \mathbf{u})\cdot \mathbf{u}=\mathbf{y}\cdot \mathbf{u}-(\alpha \mathbf{u})\cdot \mathbf{u}=\mathbf{y}\cdot \mathbf{u}-\alpha (\mathbf{u}\cdot \mathbf{u})$$

That is, (1) is satisfied with $\mathbf{z}$ orthogonal to $\mathbf{u}$ if and only if $\alpha =\frac{\mathbf{y}\cdot \mathbf{u}}{\mathbf{u}\cdot \mathbf{u}}$ and $\hat{\mathbf{y}}=\frac{\mathbf{y}\cdot \mathbf{u}}{\mathbf{u}\cdot \mathbf{u}}\mathbf{u}$. The vector $\hat{\mathbf{y}}$ is called the orthogonal projection of $\mathbf{y}$ onto $\mathbf{u}$, and the vector $\mathbf{z}$ is called the component of $\mathbf{y}$ orthogonal to $\mathbf{u}$.

This projection is determined by the subspace $L$ spanned by $\mathbf{u}$ (the line through $\mathbf{u}$ and $0$). Sometimes $\hat{\mathbf{y}}$ is denoted by $pro{j}_L\mathbf{y}$ and is called the orthogonal projection of $\mathbf{y}$ onto $L$. That is,

由正交投影可以求得点到直线的距离 ($\Vert \mathbf{y}-\hat{\mathbf{y}}\Vert$)

$\mathbf{x}\mapsto pro{j}_L\mathbf{x}$ is a linear transformation.

A Geometric Interpretation of Theorem 5

The formula for the orthogonal projection $\hat{\mathbf{y}}$ in

has the same appearance as each of the terms in Theorem 5. Thus Theorem 5 decomposes each $\mathbf{y}$ in S p a n { u 1 , . . . , u p } Span\{\boldsymbol u_1,...,\boldsymbol u_p\} Span{ u1​,...,up​} into the sum of $p$ projections onto one-dimensional subspaces that are mutually orthogonal.

Orthonormal Sets 单位正交集

A set { u 1 , . . . , u p } \{\boldsymbol u_1,...,\boldsymbol u_p\} { u1​,...,up​} is an orthonormal set if it is an orthogonal set of unit vectors. If $W$ is the subspace spanned by such a set, then { u 1 , . . . , u p } \{\boldsymbol u_1,...,\boldsymbol u_p\} { u1​,...,up​} is an orthonormal basis for $W$ , since the set is automatically linearly independent

The simplest example of an orthonormal set is the standard basis { e 1 , . . . , e n } \{\boldsymbol e_1,...,\boldsymbol e_n\} { e1​,...,en​} for ${\mathbb{R}}^n$. Any nonempty subset of { e 1 , . . . , e n } \{\boldsymbol e_1,...,\boldsymbol e_n\} { e1​,...,en​} is orthonormal, too.

When the vectors in an orthogonal set of nonzero vectors are normalized to have unit length, the new vectors will still be orthogonal, and hence the new set will be an orthonormal set.

Matrices whose columns form an orthonormal set are important in applications and in computer algorithms for matrix computations. Their main properties are given in Theorems 6 and 7.

Properties (a) and (c ) say that the linear mapping $\mathbf{x}\mapsto U\mathbf{x}$ preserves lengths and orthogonality.

Theorems 6 and 7 are particularly useful when applied to $square$ matrices. An orthogonal matrix is a square invertible matrix $U$ such that $U^{-1}=U^T$ . It is easy to see that any square matrix with orthonormal columns is an orthogonal matrix. Surprisingly, such a matrix must have orthonormal rows, too. ($(U^T{)}^{-1}=(U^T{)}^T$)

$detU=\pm 1$

EXERCISES
Show that if an $n\times n$ matrix $U$ satisfies $(U\mathbf{x})\cdot (U\mathbf{y})=\mathbf{x}\cdot \mathbf{y}$ for all $\mathbf{x}$ and $\mathbf{y}$ in ${\mathbb{R}}^n$, then $U$ is an orthogonal matrix.
SOLUTION
$U{\mathbf{e}}_j$ is the $j$-th column of $U$. Since ${\Vert U{\mathbf{e}}_j\Vert }^2=(U{\mathbf{e}}_j)\cdot (U{\mathbf{e}}_j)={\mathbf{e}}_j\cdot {\mathbf{e}}_j=1$, the columns of $U$ are unit vectors. For $j\ne k$, $(U{\mathbf{e}}_j)\cdot (U{\mathbf{e}}_k)={\mathbf{e}}_j\cdot {\mathbf{e}}_k=0$. Thus ach pair of distinct columns of $U$ is orthogonal.