A set refers to a bunch of things put together. The number field indicates that the numbers inside are closed to addition, subtraction, multiplication and division. It is a special case of sets and is infinite.

2. Linear space

Refers to a space in which any vector, whether multiplied by a constant or added, subtracted, multiplied or divided with other vectors, the result is still in this space, then this space is called a linear space. Similar to the number field concept.

3. Linear correlation and linear independence

If there is a bunch of vectors $x_{1},x_{2},x_{3}...$ such that not all $c_{1}x_{1}+c_{2}x_{2}+c_{3}x_{3}+...=0$ of them $c_{i}$ are 0, it is called $x_{i}$ linear correlation. Otherwise, it is linearly independent.

Linear correlation means that these vectors are perpendicular to each other. When each vector is not 0, the vector sum will never be 0. Linear independence means that at least one of these vectors is not perpendicular to the other vectors.

4. Bases and dimensions

A linear space can represent all vectors by a minimum of m vector combinations, then this minimum number m is the dimension of the linear space, and the m-dimensional linear space V is recorded as $V^{m}$ .

The meaning of vector coordinates: the components of a certain vector in linear space under each coordinate base are called the coordinates of the vector

5. Linear subspace

$V_{1}$ is a non-empty subset of V, which is a linear space on the number field space K, if $V_{1}$ it satisfies:

If x,y $\epsilon$ $V_{1}$ , then x+y $\epsilon$ $V_{1}$
If x $\epsilon$ $V_{1}$ , k $\epsilon$ K, then kx $\epsilon$ $V_{1}$

If $V_{1}$ it is equal to V or is the empty set, it $V_{1}$ is called a trivial subspace, otherwise it is called a non-trivial subspace.

6. Matrix range and kernel

Value range (column space): All vectors that have undergone matrix transformation, the transformed vectors fall in a space, and this space is called the value range.

Let R(A) represent the value range of matrix A. There is any vector x such that Ax=y vector. The set of y vectors is called the value range of matrix A.

$\begin{pmatrix} a_{11} &a_{12} \\ a_{21}& a_{22}\\ a_{31}&a_{32} \end{pmatrix} \begin{pmatrix} x_{1}\\ x_{2} \end{pmatrix} = \begin{pmatrix} y_{1}\\ y_{2}\\ y_{3} \end{pmatrix}$

The matrix value range satisfies:

$R(A)=\left \{Ax|x\epsilon R^{n} \right \}\subset R^{m}\\$

Kernel : After matrix transformation of any vector, the result is 0. The space composed of these vectors is the kernel space of A, or the zero space.

$N(A)=\left \{x|Ax=0 \right \}$

The dimension of the kernel space is called the zero degree of A, denoted n(A)

Dimension : The space where the kernel is located is called V. The dimension of V = the kernel space dimension of its transformation matrix + the dimension of the value domain.

$dim(V)=dim(N(A))+dim(R(A))$

Matrix 4 spaces : Any matrix can be divided into two spaces: row space + zero space or column space + left zero space

dim(nullspace) = number of columns - r = n - r

dim(left nullspace) = number of rows - r = m - r

7. Eigenvalues and eigenvectors of matrices

$Ax=\lambda x$

Vector transformation : Matrix A can be understood as the transformation of vector x, that is, after A transforms x, x is only scaled in length.

It can also be understood this way, there is a matrix A, how to find the vector x transformed by it, which only changes in length and unchanged direction. These vectors are the eigenvectors of matrix A, corresponding to the eigenvalues $\lambda$ .

All eigenvectors of the matrix form the basis of the matrix, which is the coordinate system after matrix operations. Matrix operations can be understood as changes to vectors or transformations of coordinate systems (that is, basis transformations).

Coordinate system transformation : vector x, multiplied by matrix A, becomes vector b

$Ax=b$

It can be understood that the vector remains unchanged, but the coordinate system is transformed.

x is the description under the A coordinate system, b is the description under the I coordinate system, that is, I can be regarded as omitted

$Ax = Ib$

For example: in the normal two-dimensional coordinate system, there are two ways to change the point (2,3) into the point (1,1). One is to directly move the point to the latter, that is, motion; the other is to move the point to the latter point. The horizontal axis of the coordinate system is shortened by 1/2 and the vertical axis is shortened by 1/3.

Matrix understanding : When there is a linear transformation, multiple matrices can be used to describe the transformation, but these matrices are not the transformation itself, but a way of describing the transformation, and these matrices that can describe the transformation are all similar. , have the same eigenvectors and eigenvalues.

8. Determinant

Matrix A can be understood as a linear transformation from one point to another point in n-dimensional space. If the object of transformation is also a matrix T, it can be understood as the transformation of a cube T from n-dimensional space to n-dimensional space. After transformation, it is M. The volume after transformation divided by the volume before change is the transformation rate, that is The value of the determinant.

$AT=M$

$det(A)=|A|$

Inverse of matrix : Matrix A is understood as the matrix changing from T to M, and matrix $A^{-1}$ is understood as the matrix changing from M back to T, then:

$it(A)*it(A^{-1})=1$

If the matrix is invertible, it must have a determinant, which is the transformation factor. If the matrix has no determinant, it means that the matrix must be irreversible.

9. Intersection and sum of subspaces

Intersection is intersection and sum is union.

Direct sum : $V=V_{1}+V_{2}$ The only representation is $V_{1}$ a vector sum in and $V_{2}$ a vector sum in , denoted as $V_{1}\bigoplus V_{2}$

It can be understood that a vector in a high-dimensional space is projected onto mutually orthogonal and independent dimensions. After processing them, the direct sum is the solution required by the vector, similar to the decomposition of force.

10. Rank and dimension of matrix

Rank: The rank of the matrix = the dimension of the column vector = the dimension of the row vector, which is the maximum dimension of the column vector, recorded as rank(A)

Dimension: Linear space consists of several vectors, that is, the number of column vectors, recorded as dim(A)

11.Matrix type

Common matrices and their definitions
real orthogonal matrix	$A^{T}A=I$
unitary matrix	$A^{H}A=I$
Symmetric matrix ( $AA^{T}$ for symmetric matrix)	$A=A^{T}$
Hermitian matrix	$A=A^{H}$
Normal matrix (must be similarly diagonalized)	$A^{H}A=AA^{H}$
singular matrix	$\|A\|=0$
idempotent matrix	$A^{2}=A$
simple matrix	diagonalizable matrix

11.Hermite standard type

Matrix $rank(H)=r$ , and the first r rows of H are non-zero rows and contain an identity matrix $I_{r}$ , and the last mr rows are all 0. example:

$H=\begin{pmatrix} 1 & 0 & 0 &3 \\ 0& 1& 0 & 4\\ 0& 0 & 1 & 2 \end{pmatrix}$

12.Jordan standard type

Upper triangle block, example:

$J(\lambda)=\begin{pmatrix}\lambda_{1}&1&&\\&\lambda_{2}&1&\\&&\lambda_{3}&1\\&&&\lambda _{4}\end{pmatrix}$

12. Deficiency of matrix

If the matrix has full rank, then the rank of the matrix $Rank(A)=r=n$ is $0$

If the matrix is not of full rank, the rank is $Rank(A)=r$ , and the loss is $n-r$

13.Characteristic polynomials and minimum polynomials

(1) Characteristic polynomial ${\color{Red} f(\lambda )}$

The n-order matrix A is the root (zero point) of its characteristic polynomial

$\varphi (\lambda )=det(\lambda IA)=(\lambda -\lambda _{1})(\lambda -\lambda _{2})...(\lambda -\lambda _{n})$

That is to say, it satisfies:

$\varphi (A )=(A -\lambda _{1}I)(A -\lambda _{2}I)...(A -\lambda _{n}I)$

(2) Minimum polynomial ${\color{Red} m(\lambda )}$

The characteristic polynomial with the smallest degree such that m(A)=0 has a leading coefficient of 1. The general method is to refer to it $f(\lambda )$ , lower its highest power by 1, and then try until m(A)=0 is satisfied.

14. Matrix Contract

Conditions of contract between A and B:

There exists an invertible matrix C such that $CAC=B$

15. Vector Orthogonalization

If there is a set of non-coplanar vectors, how to orthogonalize them?

The idea is to first orthogonalize two of the vectors, and then orthogonal the third component to the two orthogonal components.

16.Orthogonal complement

If vectors y and vectors are $x_{1},x_{2}...x_{n}$ both orthogonal, then $x_{1},x_{2}...x_{n}$ the linear combination of y and is also orthogonal.

w is $V^{n}$ a subspace of the Euclidean space. If there is a vector y that is orthogonal to W, then y is orthogonal to each basis vector of W.

Orthogonal complement:

y is called the orthogonal complement of W space, denoted as ${\color{Red} W^{+}}:W^{+}=\left \{ y|y\perp W,y\epsilon V^{n}\right \}$

$V^{n}=W\bigoplus W^{+}$

Dimensions of the orthogonal complement space:

$dimW=m$

$dimW^{+}=n-m$

17. Invariant factors and elementary factors

Elementary factors, invariant factors

Invariant factors:

The diagonal elements of Smith's standard type are invariant factors, arranged from small to large. respectively recorded as $d_{1},d_{2},d_{3}\cdots$

The determinant factor $D_{k}(\lambda)$ is the greatest common factor of the kth order subexpression and satisfies $d_{1}=D_{1},d_{2}=\frac{D_{2}}{D_{1}},d_{3}=\frac{D_{3}}{D_{2}}$

Elementary factors:

Remove the invariant factor constant term and split the remaining factors

example:

Invariant factors: $\lambda-1,\lambda-1,(\lambda-1)(\lambda-2)$

Elementary factors: $\lambda-1,\lambda-1,\lambda-1,\lambda-2$

18.Inverse of matrix

Inverse method of matrix 1 :

$A^{-1}=\frac{A^{*}}{|A|}$

Inverse method of matrix 2 :

Write the matrices A and I side by side, and then perform elementary row transformation at the same time to transform A into I. At this time, the position of I is. The principle is that the elementary row transformation is $A^{-1}$ equivalent to left-multiplying A by a matrix T, so that $TA=I$ , $A^{-1}=T=TI$ that is, Is equal to the matrix after I transformation.

How to find eigenvalues and eigenvectors : for any matrix A, find $|\lambda I -A|=0$

Perform row transformations and column changes so that there is only one number left in a certain row or column, and the rest are all 0, and then the determinant is equal to the value of the point multiplied by the algebraic cofactor.

Find the adjoint matrix of matrix A $A^{*}$ :

Find the algebraic cofactor at the corresponding position. What needs to be noted is that the original ${\color{Red} a_{12}}$ algebraic cofactor ${\color{Red} A_{12}}$ needs to be filled in at another position . Special attention needs to be paid here.

example:

$A=\begin{pmatrix} 1 &1 &1 \\ 1 & 0& 1\\ 0& -1 &2 \end{pmatrix}=\begin{pmatrix} a_{11} &a_{12} &a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} &a_{33} \end{pmatrix}$

$|A|=-2$

$A^{-1}=\frac{A^{*}}{|A|}=-\frac{1}{2}\begin{pmatrix} A_{11} & A_{21} & A_{31}\\ A_{12}& A_{22} & A_{32} \\ A_{13}& A_{23} & A_{33} \end{pmatrix}=\begin{pmatrix} -\frac{1}{2} & \frac{3}{2} &-\frac{1}{2} \\ 1& -1 & 0\\ \frac{1}{2}& -\frac{1}{2} & \frac{1}{2} \end{pmatrix}$

2. Two spaces

1.European style space

real inner product space

$\\ x=(a,b,c) \\ y=(m,n,k)\\ (x,y)=am+bn+ck$

The result of the inner product is a numerical value

2. Unitary space

complex inner product space

$\\ x=(a,b,c) \\ y=(m,n,k)\\ (x,y)=\bar{(y,x)}=\bar{am}+\bar{bn}+\bar{ck}$

$\\ \bar{(y,x)}$ Represents $(x,y)$ conjugation

3. Comparison between real inner product space and complex inner product space

Comparison between real inner product space and complex inner product space
	real inner product space (European space)	Complex inner product space (unitary space)
number	real numbers	plural
$(Tx,Tx)=(x,x)$	orthogonal transformation (Orthogonal matrix) $A^{T}A=I$	Unitary transformation (unitary matrix) $A^{H}A=I$
$(Tx,y)=(x,Ty)$	Symmetry transformation (real symmetry matrix) $A^{T}=A$	Hermitian transformation (unitary symmetry transformation) Hermitian matrix $A^{H}=A$
matrix eigenvalues	real numbers	real numbers
matrix eigenvector	Orthogonal	Orthogonal

3. Matrix decomposition

Purpose : Decompose the matrix into some small matrices to make it easier to calculate or analyze the characteristics of the matrix

1.Eigenvalue decomposition

$A=P\Lambda P^{-1}$

Where the invertible matrix P is equal to the orthogonal normalized combination of eigenvectors, the matrix $\Lambda = \begin{pmatrix}\lambda_{1}&&\\&\lambda_{1}&\\&&\lambda_{1}\end{pmatrix}$

tips：

The eigenvectors that make up P here do not need Schmidt orthogonalization. Schmidt orthogonalization is just for convenience $P^{-1}$ . For example, the following Schur decomposition uses Schmidt orthogonalization, because after Schmidt orthogonal normalization, P becomes an orthogonal matrix, that is $P^{T}=P^{-1}$ .

2.QR decomposition

Definition: A real (complex) non-singular matrix A can be decomposed into a normal matrix Q and a positive linear triangular matrix R

Singular matrix: a matrix with determinant = 0, that is, an irreversible matrix

The column vector of A is $a_{1},a_{2},a_{3},...a_{n}$

(1) Find the orthogonal unitization matrix of matrix A, that is, Q

Orthogonalize $a_{1},a_{2},a_{3},...a_{n}$ _

$\left\{\begin{matrix} b_{1}=a{1} & & & & \\ b_{2}=a_{2}-k_{21}b_{1}& & & & & & \\ b_{n}=a_{n}-k_{n,n-1}b_{n-1}-...-k_{n1}b_{1}& & & & & & \end{matrix}\right.$

in $k_{ij}=\frac{(a_{i},b_{j})}{(b_{j},b_{j})}(j<i)$

$\left\{\begin{matrix} a_{1}=b{1} & & & & \\ a_{2}=b_{2}+k_{21}b_{1}& & & & & & \\ a_{n}=b_{n}+k_{n,n-1}b_{n-1}+...+k_{n1}b_{1}& & & & & & \end{matrix}\right.$

$(a_{1},a_{2},a_{3},...a_{n},)=(b_{1},b_{2},b_{3},...b_{n},)C$

in $C=\begin{pmatrix} 1 &k_{21} &... &k_{n1} \\ & 1 & ... & k_{n2}\\ & & \ddots &\vdots \\ & & & 1 \end{pmatrix}$

unitization $b_{1},b_{2},b_{3},...b_{n}$ _

$q_{i}=\frac{b_{i}}{|b_{i}|}$

$Q=(q_{1},q_{2},q_{3},...q_{n})$

(2) Find the product of the orthogonal vector modulus value and C, that is, R

$R=\begin{pmatrix} |b_{1}| & & \\ & \ddots & \\ & & |b_{n}| \end{pmatrix}C=\begin{pmatrix} |b_{1}| & & \\ & \ddots & \\ & & |b_{n}| \end{pmatrix}\begin{pmatrix} 1 &k_{21} &... &k_{n1} \\ & 1 & ... & k_{n2}\\ & & \ddots &\vdots \\ & & & 1 \end{pmatrix}$

example:

$A=\begin{bmatrix} 1&2 &2 \\ 2&1 &2 \\ 2&2 &1 \end{bmatrix}$

$a_{1}=\begin{pmatrix} 1\\ 2\\ 2 \end{pmatrix} a_{2}=\begin{pmatrix} 2\\ 1\\ 2 \end{pmatrix} a_{3}=\begin{pmatrix} 2\\ 2\\ 1 \end{pmatrix}$

$\left\{\begin{matrix} b_{1}=a_{1}=\begin{pmatrix} 1\\ 2\\ 1 \end{pmatrix}\\ b_{2}=a_{2}-b_{1}=\begin{pmatrix} 1\\ -1\\ 1 \end{pmatrix}\\ b_{3}=a_{3}-\frac{(a_{3},b_{2})}{(b_{2},b_{2})}b_{2}-\frac{(a_{3},b_{1})}{(b_{1},b_{1})}b_{1}=\begin{pmatrix} 0.5\\ 0\\ -0.5 \end{pmatrix} \end{matrix}\right.$

unitization $b_{1},b_{2},b_{3}$ _

$Q=\begin{pmatrix} \frac{1}{\sqrt{6}} &\frac{1}{\sqrt{3}} &\frac{1}{\sqrt{2}} \\ \frac{2}{\sqrt{6}} &-\frac{1}{\sqrt{3}} & 0\\ \frac{1}{\sqrt{6}} & \frac{1}{\sqrt{3}} &-\frac{1}{\sqrt{2}} \end{pmatrix}$

$R=\begin{pmatrix} \sqrt{6} & & \\ & \sqrt{3} & \\ & & \frac{1}{\sqrt{2}} \end{pmatrix}\begin{pmatrix} 1 & 1 & \frac{7}{6}\\ &1 &\frac{1}{3} \\ & &1 \end{pmatrix}=\begin{pmatrix} \sqrt{6} &\sqrt{6} &\frac{7}{\sqrt{6}} \\ & \sqrt{6}& \frac{1}{\sqrt{3}}\\ & & \frac{1}{\sqrt{2}} \end{pmatrix}$

Givens matrix and Givens transformation:

The Givens matrix is an elementary rotation matrix, and the transformation determined by the Givens matrix is called the Givens transformation, which is an elementary rotation transformation.

nature:

(1) Givens matrix is an orthogonal matrix

(2) Any invertible matrix can be transformed into an upper triangular matrix by left-multiplying a limited number of Givens matrices.

Householder matrix:

The Householder matrix is also called the elementary reflection matrix, which is equivalent to mirroring the vector or matrix, and the modulus value remains unchanged.

nature:

(1) $\\A^{T}=A \\A^{T}A=I$ Belongs to a symmetric orthogonal matrix

（2） $\\A^{2}=I \\A^{-1}=A\\ detA=-1$

(3) The Givens matrix is the product of two Householder matrices

(4) Any reversible matrix can be transformed into an upper triangular matrix by left-multiplying a limited number of Householder matrices

3. Normal matrix and Schur decomposition

Schur’s Lemma:

$A\epsilon R^{n\ast n}$ And the eigenvalues of A are all real numbers, then there is an orthogonal matrix Q such that

$Q^{T}AQ=\begin{pmatrix} \lambda _{1} &\cdots & \cdots&*\\ &\lambda _{2} &\ddots &\vdots\\ & & \ddots &\vdots \\ &&&\lambda _{n} \end{pmatrix}=Q^{-1}AQ$

Any real square matrix A is orthogonally similar to an upper triangular matrix, and its main diagonal elements are the eigenvalues of matrix A.

Schur decomposition process :

The principle is that it is known $P^{-1}AP=\Lambda$ that P is a full-rank matrix. In fact, when P is a unitary matrix, here $P^{-1}=P^{H}$ , therefore, finding the unitary matrix P is equivalent to finding the full-rank matrix P before.

example:

$A=\begin{pmatrix} 4+3i &4i &-6-2i \\ -4i &4-3i &-2-6i \\ 6+2i&-2-6i &1 \end{pmatrix}$

Find the unitary matrix Q such that it $Q^{H}AQ$ is a diagonal matrix

(1) Find the eigenvector corresponding to the eigenvalue

a. Find eigenvalues

$|\lambda I-A|=\begin{vmatrix} \lambda -4-3i &-4i &6+2i \\ 4i & \lambda -4+3i &2+6i \\ -6-2i&2+6i & \lambda -1 \end{vmatrix}=0$

$\\ \lambda _{1}=-9i \\ \lambda _{2}= 9i \\ \lambda _{3}= 9$

$\lambda _{1}=-9i$ at that time

$x_{1}=\begin{pmatrix} -\frac{i}{2}\\ 1\\ 1 \end{pmatrix}$

$\lambda _{2}=9i$ at that time

$x_{2}=\begin{pmatrix} -i\\ -\frac{1}{2}\\ 1 \end{pmatrix}$

$\lambda _{3}=9$ at that time

$x_{3}=\begin{pmatrix} i\\ 1\\ -\frac{1}{2} \end{pmatrix}$

(2) Orthogonally normalize the feature vectors

$\eta _{1}=\begin{pmatrix} -\frac{i}{3}\\ \frac{2}{3}\\ \frac{2}{3} \end{pmatrix}$

$\eta _{2}=\begin{pmatrix} \frac{2i}{3}\\ -\frac{1}{3}\\ \frac{2}{3} \end{pmatrix}$

$\eta _{3}=\begin{pmatrix} \frac{2i}{3}\\ \frac{2}{3}\\ -\frac{1}{3} \end{pmatrix}$

（3） $Q=(\eta _{1},\eta _{2},\eta _{3})$

4. Full rank decomposition

Definition : If $A\epsilon C_{r}^{m*n}$ , if there exists $F\epsilon C_{r}^{m*r}$ a sum such $G\epsilon C_{r}^{r*n}$ that , then $A=FG$ it is called a full-rank decomposition of a matrix. Full rank decomposition is not unique

example:

$A=\begin{pmatrix} 0 &0 &1 \\ 2& 1 &1 \\ 2i& i & 0 \end{pmatrix}$

$A\Rightarrow \begin{pmatrix} 1 &\frac{1}{2} &0 \\ 0& 0& 1\\ 0 &0 &0 \end{pmatrix}$

$A=\begin{pmatrix} 0&1 \\ 2 &1 \\ 2i& 0 \end{pmatrix} \begin{pmatrix} 1& \frac{1}{2} &0 \\ 0& 0& 1 \end{pmatrix}=FG$

G is the simplest form

5. Singular value decomposition

On the basis of eigenvalue decomposition, singular value decomposition is derived. The basic principle is the same, except that eigenvalue decomposition corresponds to an $n*n$ order matrix, and singular value decomposition corresponds to $m*n$ an order matrix, not a square matrix.

For $m*n$ the order matrix A, $AA^{H}$ which is $m*m$ an order symmetric matrix (symmetric matrix $A=A^{T}$ ), $A^{H}A$ and is $n*n$ an order symmetric matrix, then we have

$AA^{H}=P\Lambda P^{H}$

$A^{H}A=Q\Lambda Q^{H}$

P is $AA^{H}$ the left singular vector of , Q is $A^{H}A$ the right singular vector of

Both P and T are unitary matrices, and unitary matrices are also orthogonal matrices. The diagonalization of the orthogonal matrix can be written as $A=R\Lambda R^{T}$

but ${\color{Red} A=PS_{r} Q^{H}}$

$S_{r}$ Taking the square root of the eigenvalues of A is called the singular value of A

example:

$A=\begin{pmatrix} 1 &0 &1 \\ 0 & 1 &1 \\ 0&0 &0 \end{pmatrix}$

（1） $AA^{T}=V\Sigma V^{T}$

$A^{T}A=\begin{pmatrix} 2 &1 &0 \\ 1 & 2 & 0\\ 0& 0&0 \end{pmatrix}$

Since it is not of full rank, only the square matrix in the upper left corner is calculated. The eigenvalues are $\\ \lambda _{1}=3\\ \lambda _{2}=1$ , $\lambda _{3}=0$

The corresponding feature vector is:

$p_1=\begin{pmatrix} 1\\ 1\\ 0 \end{pmatrix} p_2=\begin{pmatrix} 1\\ -1\\ 0 \end{pmatrix} p_3=\begin{pmatrix} 0\\ 0\\ 1 \end{pmatrix}$

After unitization:

$U=\begin{pmatrix} \frac{1}{\sqrt{2}} &\frac{1}{\sqrt{2}} &0 \\ \frac{1}{\sqrt{2}} &-\frac{1}{\sqrt{2}} &0 \\ 0 & 0 & 1 \end{pmatrix}$

（2） $A^{T}A=U\Sigma U^{T}$

$A^{T}A=\begin{pmatrix} 1 &0 &1 \\ 0& 1 & 1\\ 1& 1& 2 \end{pmatrix}$

The eigenvalues are: $\\ \lambda _{1}=3 \\\lambda _{2}=1 \\ \lambda _{3}=0$

The eigenvector is: $p_1=\begin{pmatrix} 1\\ 1\\ 2 \end{pmatrix} p_2=\begin{pmatrix} 1\\ -1\\ 0 \end{pmatrix} p_3=\begin{pmatrix} 1\\ 1\\ -1 \end{pmatrix}$

After unitization:

$V=\begin{pmatrix} \frac{1}{\sqrt{6}} &\frac{1}{\sqrt{2}} &\frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{6}} &-\frac{1}{\sqrt{2}} &\frac{1}{\sqrt{3}} \\ \frac{2}{\sqrt{6}} & 0 & -\frac{1}{\sqrt{3}} \end{pmatrix}$ .

（3）

$\Sigma =\begin{pmatrix} 3 & & \\ & 1 & \\ & &0 \end{pmatrix}$

${\color{Red} AV=US_{r}}$

Solution ideas:

The solution process of singular value decomposition (SVD) is actually extremely simple, and $A=US_{r}V^{T}$ the following two formulas can be derived:

$\\AA^{T}=US_{r}V^{T}(US_{r}V^{T})^{T}=US_{r}V^{T}VS_{r}U^{T}=U\Lambda U^{T}\\ A^{T}A=(US_{r}V^{T})^{T}US_{r}V^{T}=VS_{r}U^{T}US_{r}V^{T}=V\Lambda V^{T}\\$

$S_{r}=\sqrt{\Lambda }$

So the singular value decomposition is actually very simple. Treat $AA^{T}$ and $A^{T}A$ as a square matrix for shur decomposition, that is, find the standard orthonormal basis corresponding to the eigenvalue. If A is an m*n order matrix, it $AA^{T}$ corresponds to U and $A^{T}A$ the solution to V, and they eigenvalues are the same.

6. Spectral decomposition

Definition: A simple matrix (diagonalizable matrix) A can be decomposed into the sum of the eigenvalues and the idempotent matrix product, that is

$A=\sum_{i=1}^{k}\lambda _{i}E_{i}$

$\lambda _{i}$ is called the spectral value of A, that is, the eigenvalue of A, and $E_{i}$ is called the spectral matrix of A.

Therefore, spectral decomposition can actually be regarded as the eigenvalue decomposition of a matrix, which decomposes the matrix into the sum of the products of a single eigenvalue and a single matrix.

Eigenvalue decomposition, recorded as $A=P\Lambda P^{-1}$

$P=\begin{pmatrix} \alpha _{1} \\ \alpha_{2} \\ \alpha_{3} \end{pmatrix} =\begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} &a_{32} & a_{33} \end{pmatrix}$

$P^{-1}=\begin{pmatrix} \beta _{1} &\beta _{2} & \beta _{3} \end{pmatrix} =\begin{pmatrix} b_{11} & b_{12} & b_{13} \\ b_{21} & b_{22} & b_{23} \\ b_{31} & b_{32} & b_{33} \end{pmatrix}$

$A=P\Lambda P^{-1}=\begin{pmatrix} \alpha_{1} \\ \alpha_{2} \\ \alpha_{3} \end{pmatrix} \begin{pmatrix} \lambda _{1} & & \\ &\lambda _{2} & \\ & & \lambda _{3} \end{pmatrix} \begin{pmatrix} \beta _{1}& \beta _{2}&\beta _{3} \end{pmatrix} = \lambda _{1}\alpha_{1}\beta _{1} + \lambda _{2}\alpha_{2} \beta _{2}+ \lambda _{3} \alpha_{3}\beta _{3}=\sum_{i=1}^{n}\lambda _{i}\alpha_{i} \beta _{i}$

Solution ideas :

(1) Find eigenvalues and eigenvectors

(2) Normalize the feature vector, that is, find the sum of P $P^{-1}$

(3) Decompose P and $P^{-1}$ find $\sum_{i=1}^{n}\lambda _{i}\alpha_{i} \beta _{i}$

example:

$A=\begin{pmatrix} 1 & 2 & 2\\ 2 & 1& 2\\ 2 &2 & 1 \end{pmatrix}$

The eigenvalues are respectively $\lambda _{1}=\lambda _{2}=-1$ , $\lambda _{3}=5$

$P=\begin{pmatrix} 1 & 1 &1 \\ -1& 0& 1\\ 0& -1& 1 \end{pmatrix}$

$P^{-1}=\begin{pmatrix} \frac{1}{3} & -\frac{2}{3} &\frac{1}{3} \\ \frac{1}{3} & \frac{1}{3} &-\frac{2}{3} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \end{pmatrix}$

$E_{1}=\begin{pmatrix} 1 & 1\\ -1 & 0\\ 0&-1 \end{pmatrix} \begin{pmatrix} \frac{1}{3}& -\frac{2}{3} & \frac{1}{3}\\ \frac{1}{3}& \frac{1}{3} & - \frac{2}{3} \end{pmatrix}=\begin{pmatrix} \frac{2}{3} &-\frac{1}{3} &-\frac{1}{3} \\ - \frac{1}{3}& \frac{2}{3} &-\frac{1}{3} \\ -\frac{1}{3}&-\frac{1}{3} & \frac{2}{3} \end{pmatrix}$

$E_{2}=\begin{pmatrix} 1\\ 1\\ 1 \end{pmatrix}\begin{pmatrix} \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \end{pmatrix}=\begin{pmatrix} \frac{1}{3} & \frac{1}{3} & \frac{1}{3}\\ \frac{1}{3}& \frac{1}{3}& \frac{1}{3}\\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \end{pmatrix}$

$A=-1E_{1}+5E_{2}$

4. Generalized inverse matrix

$\\ A^{-1}AA^{-1}=A^{-1}\\ AA^{-1}A=A$

Corresponds to the following one-two-two formulas.

Assume a matrix $A\epsilon C^{m*n}$ , if the matrix $X\epsilon C^{n*m}$ satisfies one or all of the following

$\\ AXA=A \\ XAX=X \\ (AX)^{H}=AX\\ (XA)^{H}=XA$

Then X is called the generalized inverse matrix of A. If all four are satisfied, X is called the Penrose inverse of A, recorded as $A^{+}$

1. Find the generalized inverse of full rank decomposition

If $A=FG$ , then

$A^{+}=G^{H}(GG^{H})^{-1}(F^{H}F)^{-1}F^{H}$

Therefore, it is necessary to first find the full rank decomposition $A=FG$

2. Find the generalized inverse using singular value decomposition

If $A=PS_{r} Q^{H}$ , $Q=(Q_{1},Q_{2})$ then

$A^{+}=Q_{1}\Sigma ^{-1}Q_{1}^{H}A^{H}$

3. Spectral decomposition to find generalized inverse

4. The system of equations is compatible

Definition : There exists a system of non-homogeneous linear equations $Ax=b$

If $r(A)=r(A\vdots b)$ , then this system of linear equations is said to be consistent

5.{1}-reverse

$Ax=b$

$x=A^{(1)}b+(I-A^{(1)}A)y$

6. Whether the generalized inverse judgment equation has a solution

Determine $Ax=b$ whether there is a solution

If $AA^{+}b=b$ , then there is a solution

5. Vector and matrix norms

$Ax=b$ , the norm of vector x represents the size of the vector, and the norm of matrix A represents the rate of change when changing from vector x to vector b, that is, the size of the scaling amount. The norm is a size measurement tool.

1. Vector norm

category	official	meaning
1-norm	$\left \\| x \right \\|_{1}=\sum_{i=1}^{n}\|x_{i}\|$	sum of absolute values of x
2-norm	$\left \\| x \right \\|_{2}=\sqrt{\sum_{i=1}^{n}\|x_{i}\|^{2}}$	xEuclidean distance
$\infty$ -Norm	$\left \\| x \right \\|_{\infty }=max\|x_{i}\|$	The largest absolute value of x
p-norm	$\left \\| x \right \\|_{p}={\sum_{i=1}^{n}\|x_{i}\|^{p}}^{\frac{1}{p}}$	The pth power and the root $\frac{1}{p}$ power of x
E-norm	$\left \\| x \right \\|_{E}=\sqrt{\sum_{i=1}^{n}x_{i}^{2}}$	The absolute value differs from the 2-norm

2.Matrix norm

category	official	meaning
1-norm	$\left \\| A \right \\|_{1}=\underset{j}{max}\sum_{i=1}^{n}\|a_{ij}\|$	The sum of the absolute values of the column elements, taking the maximum
2-norm	$\left \\| A \right \\|_{2}=\sqrt{max(\lambda _{i})}$	$A^{T}A$ Maximum eigenvalue square root
$\infty$ -Norm	$\left \\| A \right \\|_{\infty }=\underset{i}{max}\sum_{j=1}^{n}\|a_{ij}\|$	Corresponds to 1-norm, the maximum absolute value sum of row elements
F-norm	$\|\|A\|\|_{F}=(\sum_{i=1}^{m}\sum_{j=1}^{n}\|a_{ij}\|^{2})^{\frac{1}{2}}$	Sum of squares of all elements

3. Solutions to non-homogeneous linear equations

$Ax=b$

$x=A^{(1)}b+(I-A^{(1)}A)y$

because $A^{+}\epsilon A^{(1)}$ , so

$x=A^{+}b+(I-A^{+}A)y$

If row A has full rank, then:

$A^{+}=A^{H}(AA^{H})^{-1}$

If column A has full rank, then:

$A^{+}=(A^{H}A)^{-1}A^{H}$

$A^{+}=G^{H}(GG^{H})^{-1}(F^{H}F)^{-1}F^{H}$

6. Matrix sequences and matrix functions

1.Matrix function

Assume that the one-variable function f(z) can be expanded into a power series of z $f(z)=\sum_{k=0}^{\infty }c_{k}z^{k}(|z|<r)$

silu $r>0$ represents the convergence radius of the power series. When the spectral radius of the n-order matrix A is $\rho (A)<r$ , the sum of the converged matrix power series is $f(z)=\sum_{k=0}^{\infty }c_{k}A^{k}$ called the matrix function, recorded as

$f(A)=\sum_{k=0}^{\infty }c_{k}A^{k}$

z	A
$e^{z}=1+\frac{z}{1!}+\frac{z^{2}}{2!}+\frac{z^{3}}{3!}\cdots$	$e^{A}=I+\frac{A}{1!}+\frac{A^{2}}{2!}+\frac{A^{3}}{3!}\cdots$
$cos z=1-\frac{z^{2}}{2!}+\frac{z^{4}}{4!}\cdots$	$cos A=I-\frac{A^{2}}{2!}+\frac{A^{4}}{4!}\cdots$
$sin z=1-\frac{z^{3}}{3!}+\frac{z^{5}}{5!}\cdots$	$sin A=I-\frac{A^{3}}{3!}+\frac{A^{5}}{5!}\cdots$

$\sum_{k=1}^{\infty }kA^{k}=(I-A)^{-2}A$

$\sum_{k=1}^{\infty }k^{2}A^{k}=(I-A)^{-2}[(I-A)^{-1}-2I-A]$

2. Solving matrix functions

1. Undetermined coefficient method

Solution ideas:

First find the minimum characteristic polynomial $m_{A}(\lambda )$ , and then assume that the matrix function to be found is $f(\lambda)$ , $f(\lambda)=m_{A}( \lambda )*g(\lambda)+r(\lambda)$ , where it $r(\lambda)$ is $m_{A}(\lambda)$ a polynomial with degree less than . Then bring in the eigenvalues $f(\lambda)$ one by one $\lambda_{i}$ , and if there are multiple roots, take $f(\lambda)$ the derivative.

example:

$A=\begin{pmatrix} 2 &0 &0 \\ 1& 1& 1\\ 1&-1 & 3 \end{pmatrix}$ ,beg $sinAt(t\epsilon R)$

$det(\lambda I-A)=(\lambda-2)^{3}$

$m(\lambda)=(\lambda-2)^{2}$

$f(\lambda)=sin\lambda t=m(\lambda)g(\lambda)+r(\lambda)=m(\lambda)g(\lambda)+a\lambda+b$

$\begin{cases} &f(2)=sin2t=2a+b \\ & f^{'}(2)=tcos2t=a \end{cases}$ have to $\begin{cases} &a=tcos2t \\ & b=sin2t-2tcos2t \end{cases}$

$r(\lambda)=(tcos2t)\lambda+sin2t-2tcos2t$

$sinAt=tcos(2t)A+(sin2t-2tcos2t)I=\begin{pmatrix} tsin2t &0 &0 \\ tcos2t& sin2t-tcos2t & tcos2t\\ tcos2t& -tcos2t &sin2t+tcos2t \end{pmatrix}$

2.Jordan standard method

3. Matrix differential equations

(1) Homogeneous differential equation

$x^{'}=\frac{dx}{dt}=Ax$

$x=e^{tA}x(0)$

(2) Non-homogeneous differential equations

$x^{'}=\frac{dx}{dt}=Ax+b(t)$

$x=e^{tA}x(0)+e^{tA}\int_{t_{0}}^{t}e^{-st}b(s)ds$

example:

$A=\begin{pmatrix} 2 &0 &0 \\ 1 & 1 & 1\\ 1&-1 &3 \end{pmatrix}$

$b(t)=\begin{pmatrix} e^{2t}\\ e^{2t}\\ 0 \end{pmatrix}$

$x(0)=\begin{pmatrix} -1\\ 1\\ 0 \end{pmatrix}$

Solve differential equations $x^{'}=Ax+b(t)$ that satisfy x(0)

${\color{Blue} x=e^{tA}x(0)+e^{tA}\int_{t_{0}}^{t}e^{-st}b(s)ds=e^{tA}[x(0)+\int_{t_{0}}^{t}e^{-st}b(s)ds]}$

Let’s find a general solution first:

Seek first $e^{tA}$

$m(\lambda)=(\lambda-2)^{2}$

$f(\lambda)=e^{t\lambda}=m(\lambda)g(\lambda)+r(\lambda)=m(\lambda)g(\lambda)+a\lambda+b$

$\begin{cases} &f(2)=e^{2t}=2a+b \\ & f^{'}(2)=te^{2t}=a \end{cases}$ have to $\begin{cases} &a=te^{2t} \\ & b=e^{2t}-2te^{2t} \end{cases}$

$r(\lambda)=te^{2t}\lambda+e^{2t}-2te^{2t}$

$e^{tA}=te^{2t}A+(e^{2t}-2te^{2t})I$

$e^{tA}=e^{2t}\begin{pmatrix} 1 & 0&0 \\ t & 1-t& t\\ t &-t &1+t \end{pmatrix}$

$x(0)=\begin{pmatrix} -1\\ 1\\ 0 \end{pmatrix}$

x(0) is c

$e^{tA}x(0)=e^{tA}\begin{pmatrix} -1\\ 1-2t\\ -2t \end{pmatrix}$

Looking for a special solution:

$\int_{t_{0}}^{t}e^{-st}b(s)ds=\int_{t_{0}}^{t}\begin{pmatrix} 1\\ 1\\ 0 \end{pmatrix}ds=\begin{pmatrix} t\\ t\\ 0 \end{pmatrix}$

$e^{-st}=-te^{-2t}A+(e^{-2t}+2te^{-2t})=e^{-2t}\begin{pmatrix} 1 & 0&0 \\ -t & 1+t& -t\\ -t &t &1-t \end{pmatrix}$

merge:

$x=e^{tA}[\begin{pmatrix} -1\\ 1\\ 0 \end{pmatrix} +\begin{pmatrix} t\\ t\\ 0 \end{pmatrix}]=e^{tA}\begin{pmatrix} t-1\\ 1+t\\ 0 \end{pmatrix}=e^{2t}\begin{pmatrix} 1 & 0&0 \\ t & 1-t& t\\ t &-t &1+t \end{pmatrix}\begin{pmatrix} t-1\\ 1+t\\ 0 \end{pmatrix}=e^{2t}\begin{pmatrix} t-1\\1-t \\ -2t\end{pmatrix}$

7. Matrix eigenvalue estimation

(1) Line Gaelic Circle

Except for the diagonal elements in each row, the sum of the absolute values of the other elements is the boundary of the radius of the first Gaelic circle, and the center of the circle is the diagonal element.

(2) Leger circle

The principles of the column Gael circle and row Gael circle are the same.

(3) Gaelic circle isolation

$D=\begin{pmatrix} 1 & & \\ & 1& \\ & & n \end{pmatrix}$

$DAD^{-1}=\begin{pmatrix} & & \frac{1}{n}\\ & & \frac{1}{n}\\ n & n&1 \end{pmatrix}$

Basics of Matrix Theory

Reference blog

1. Linear space

1. Sets and number fields

2. Linear space

3. Linear correlation and linear independence

4. Bases and dimensions

5. Linear subspace

6. Matrix range and kernel

7. Eigenvalues ​​and eigenvectors of matrices

8. Determinant

9. Intersection and sum of subspaces

10. Rank and dimension of matrix

11.Matrix type

11.Hermite standard type

12. Deficiency of matrix

13.Characteristic polynomials and minimum polynomials

14. Matrix Contract

15. Vector Orthogonalization

16.Orthogonal complement

17. Invariant factors and elementary factors

18.Inverse of matrix

2. Two spaces

1.European style space

2. Unitary space

3. Comparison between real inner product space and complex inner product space

3. Matrix decomposition

1.Eigenvalue decomposition

2.QR decomposition

3. Normal matrix and Schur decomposition

4. Full rank decomposition

5. Singular value decomposition

6. Spectral decomposition

4. Generalized inverse matrix

1. Find the generalized inverse of full rank decomposition

2. Find the generalized inverse using singular value decomposition

3. Spectral decomposition to find generalized inverse

4. The system of equations is compatible

5.{1}-reverse

6. Whether the generalized inverse judgment equation has a solution

5. Vector and matrix norms

1. Vector norm

2.Matrix norm

3. Solutions to non-homogeneous linear equations

6. Matrix sequences and matrix functions

1.Matrix function

2. Solving matrix functions

1. Undetermined coefficient method

2.Jordan standard method

3. Matrix differential equations

7. Matrix eigenvalue estimation

Guess you like

7. Eigenvalues and eigenvectors of matrices