Linear Algebra | Mathematics Fundamentals for Machine Learning

foreword

Linear algebra is a branch of mathematics concerned with vector spaces and linear maps. It includes the study of lines, surfaces, and subspaces, as well as the general properties of all vector spaces.

This article mainly introduces the core basic concepts of linear algebra used in machine learning , for readers to check for gaps in the learning stage or for quick learning reference .

Linear Algebra

determinant

1. Determinant by row (column) expansion theorem

(1) 设$A=(a_{ij}{)}_{n\times n}$，则：$a_i{A}_j+a_{i2}{A}_j+\cdots +a_{in}{A}_{jn}=\{\begin{array}{l}|A|,i=j\\ 0,i\ne j\end{array}$

或$a_1{A}_1+a_2{A}_2+\cdots +a_{}{A}_{nj}=\{\begin{array}{l}|A|,i=j\\ 0,i\ne j\end{array}$即 $A{A}^{\ast }=A^{\ast }A=\left|A\right|E,$其中：$A^{\ast }=\left(\begin{array}{cccc}{A}_{11}& A_{12}& \dots & A_{1n}\\ A_{21}& A_{22}& \dots & A_2\\ \dots & \dots & \dots & \dots \\ A_n& A_n& \dots & A_{nn}\end{array}\right)=(A_{ji})={\left(A_{ij}\right)}^T$

$D_n=\left|\begin{array}{cccc}1& 1& \dots & 1\\ x_1& x_2& \dots & x_n\\ \dots & \dots & \dots & \dots \\ x_1^{n-1}& x_2^{n-1}& \dots & x_n^{n-1}\end{array}\right|={\prod }_{1\le j<i\le n}^{}(x_i-x_j)$

(2) $A,B$ is$n$阶方阵，则$\left|AB\right|=\left|A\right|\left|B\right|=|B||A|=\left|BA\right|$，但$\left|A\pm B\right|=|A|\pm |B|$ does not necessarily hold.

(3) $\left|kA\right|=k^n\left|A\right|$,$A$ is$n$ order square matrix.

(4) $A$ is$n$阶方阵，$|A^T|=|A|;|A^{-1}|=|A{|}^{-1}$ (young$A$ reversible),$|A^{\ast }|=|A{|}^{n-1}$

$n\ge 2$

(5) $\left|\begin{array}{cc}& AO\\ & OB\end{array}\right|=\left|\begin{array}{cc}& AC\\ & OB\end{array}\right|=\left|\begin{array}{cc}& AO\\ & CB\end{array}\right|=|A||B|$
，$A,B$为方阵，但$\left|\begin{array}{cc}O& A_{m\times m}\\ B_{n\times n}& O\end{array}\right|=({-1)}^{mn}|A||B|$ 。

(6) 范德蒙行列式$D_n=\left|\begin{array}{cccc}1& 1& \dots & 1\\ x_1& x_2& \dots & x_n\\ \dots & \dots & \dots & \dots \\ x_1^{n-1}& x_2^n& \dots & x_n^{n-1}\end{array}\right|={\prod }_{1\le j<i\le n}^{}(x_i-x_j)$

AA $A$ is$n$阶方阵，$l_i(i=1,2\cdots ,n)$ is$A$ 's$n$ special attacks, 则
$|A|={\prod }_{i=1}^n{l}_i$

matrix

Matrix: $m\times n$ number$a_{ij}$Arranged in $m$ row$n$列的表格$\left[\begin{array}{c}{a}_{11}{a}_{12}\cdots a_{1n}\\ a_{21}{a}_{22}\cdots a_2\\ \cdots \cdots \cdots \cdots \cdots \\ a_m{a}_m\cdots a_{mn}\end{array}\right]$called matrix, abbreviated as $A$，或者${\left(a_{ij}\right)}_{m\times n}$. If $m=n$ , it is called$A$ is$matrix\; of\; order\; n$ or$n$ order square matrix.

Linear operations on matrices

1. Matrix addition

设$A=(a_{ij}),B=(b_{ij})$ are two$m\times n$ matrix, then$m\times n$ matrixC$C=c_{ij})=a_{ij}+b_{ij}$called matrix $A$ and$The\; sum\; of\; B$ , denoted as$A+B=C$ 。

2. Multiplication of matrices

任$A=(a_{ij})$ is$m\times n$ -matrix,$k$ is a constant, then$m\times n$ matrix$\left(k{a}_{ij}\right)$ is called the number$k$ and matrix$The\; number\; multiplication\; of\; A$ , denoted as$kA$ .

3. Multiplication of matrices

任$A=\left(a_{ij}\right)$ is$m\times n$ matrix,$B=(b_{ij})$ is$n\times s$ matrix, then$m\times s-$ matrix$C=(c_{ij})$，其中$c_{ij}=a_i{b}_1+a_{i2}{b}_2+\cdots +a_{in}{b}_{nj}={\sum }_{k=1}^n{a}_{}{b}_{kj}$called $The\; product\; of\; AB$ is written as$C=AB$ 。

4. ${\mathbf{A}}^{\mathbf{T}}$、$A^{-1}$、$A^{\ast }$ Relationship between the three

(1) ${(A^T)}^T=A,{(AB)}^T=B^T{A}^T,{\left(kA\right)}^T=to{A}^T,{(A\pm B)}^T=A^T\pm B^T$

(2) ${\left(A^{-1}\right)}^{-1}=A,{\left(AB\right)}^{-1}=B^{-1}{A}^{-1},{\left(kA\right)}^{-1}=\frac{1}{k}{A}^{-1},$

However, ${(A\pm B)}^{-1}=A^{-1}\pm B^{-1}$ does not necessarily hold.

(3) ${\left(A^{\ast }\right)}^{\ast }=|A{|}^{n-2}A(n\ge 3)$，${\left(AB\right)}^{\ast }=B^{\ast }{A}^{\ast },$ ${\left(kA\right)}^{\ast }=k^{n-1}{A}^{\ast }\left(n\ge 2\right)$

但${\left(A\pm B\right)}^{\ast }=A^{\ast }\pm B^{\ast }$ Not necessarily established.

(4) ${(A^{-1})}^T={(A^T)}^{-1},{\left(A^{-1}\right)}^{\ast }={(A{A}^{\ast })}^{-1},{\left(A^{\ast }\right)}^T={\left(A^T\right)}^{\ast }$

5.有关${\mathbf{A}}^{\ast }$ Conclusion

(1) $A{A}^{\ast }=A^{\ast }A=|A|E$

(2) $|A^{\ast }|=|A{|}^{n-1}(n\ge 2),{\left(kA\right)}^{\ast }=k^{n-1}{A}^{\ast },{\left(A^{\ast }\right)}^{\ast }=|A{|}^{n-2}A(n\ge 3)$

(3) Young $A$ reversible,$A^{\ast }=|A|A^{-1},{(A^{\ast })}^{\ast }=\frac{1}{|A|}A$

(4) Young $A$ is$n$ order square matrix, then:

$r(A^{\ast })=\{\begin{array}{l}n,r(A)=n\\ 1,r\left(A\right)=n-1\\ 0,r\left(A\right)<n-1\end{array}$

6.有关$A^{-1}$ conclusion

$A$可逆$\iff AB=E;\iff |A|\ne 0;\iff r(A)=n;$

$\iff A$ can be expressed as the product of elementary matrices;$\iff A;\iff Ax=0$。

7. Conclusions about the rank of matrices

(1) Order $r\left(A\right)$ = row rank = column rank;

(2) $r(A_{m\times n})\le \min (m,n);$

(3) $A\ne 0\Rightarrow r(A)\ge 1$；

(4) $r(A\pm B)\le r\left(A\right)+r(B);$

(5) Elementary transformation does not change the rank of the matrix

(6) $r\left(A\right)+r(B)-n\le r(AB)\le \min (r(A),r(B)),$ especially if$AB=O$
则：$r(A)+r(B)\le n$

(7) Young $A^{-1}$存在$\Rightarrow r(AB)=r\left(B\right);$ if$B^{-1}$存在
$\Rightarrow r\left(AB\right)=r(A);$

若$r(A_{m\times n})=n\Rightarrow r(AB)=r(B);$ 若$r\left(A_{m\times s}\right)=n\Rightarrow r\left(AB\right)=r\left(A\right)$。

(8) $r\left(A_{m\times s}\right)=n\iff Ax=0$ has only zero solution

8. Block inversion formula

${\left(\begin{array}{cc}A& O\\ O& B\end{array}\right)}^{-1}=\left(\begin{array}{cc}{A}^{-1}& O\\ O& B^{-1}\end{array}\right)$； ${\left(\begin{array}{cc}A& C\\ O& B\end{array}\right)}^{-1}=\left(\begin{array}{cc}{A}^{-1}& -A^{-1}C{B}^{-1}\\ O& B^{-1}\end{array}\right)$；

${\left(\begin{array}{cc}A& O\\ C& B\end{array}\right)}^{-1}=\left(\begin{array}{cc}{A}^{-1}& O\\ -B^{-1}C{A}^{-1}& B^{-1}\end{array}\right)$； ${\left(\begin{array}{cc}O& A\\ B& O\end{array}\right)}^{-1}=\left(\begin{array}{cc}O& B^{-1}\\ A^{-1}& O\end{array}\right)$

ThisriAA $A$，$B$ is a reversible square matrix.

vector

1. Linear representation of vector groups

(1) $a_1,a_2,\cdots ,a_s$Linear correlation $\iff$ At least one vector can be linearly represented by the rest.

(2) $a_1,a_2,\cdots ,a_s$Linearly independent, $a_1,a_2,\cdots ,a_s$, $\beta$ linear correlation$\iff \beta$ can be composed of$a_1,a_2,\cdots ,a_s$Unique linear representation.

(3) $\beta$ can be composed of$a_1,a_2,\cdots ,a_s$linear representation⇔
$\iff r(a_1,a_2,\cdots ,a_s)=r(a_1,a_2,\cdots ,a_s,b)$ _

2. On the linear dependence of vector groups

(1) The part is related and the whole is related; the whole is irrelevant and the part is irrelevant.

(2) ① $n$ nn$n-$ dimensional vector
$a_1,a_2\cdots a_n$∣ [ α 1 α 2 ⋯ α n ] ∣ ≠ 0 \Leftrightarrow \left|\left\lbrack \alpha_{1}\alpha_{2}\cdots\alpha_{n} \right\rbrack \right | $\iff |\left[a_1{a}_2\cdots a_n\right]|\ne 0$， $n$ nn$n-$ dimensional vector$a_1,a_2\cdots a_n$Linear correlation
$\iff |[a_1,a_2,\cdots ,a_n]|=0$
。

② $n+1$ nn_$n-$ dimensional vectors are linearly related.

③ 若$a_1,a_2\cdots a_S$Linearly independent, it is still linearly independent after adding components; or a set of vectors is linearly dependent, and still linearly dependent after removing some components.

3. Linear representation of vector groups

(1) $a_1,a_2,\cdots ,a_s$Linear correlation $\iff$ At least one vector can be linearly represented by the rest.

(2) $a_1,a_2,\cdots ,a_s$Linearly independent, $a_1,a_2,\cdots ,a_s$, $\beta$ linear correlation$\iff \beta$ can be composed of$a_1,a_2,\cdots ,a_s$Unique linear representation.

(3) $\beta$ can be composed of$a_1,a_2,\cdots ,a_s$linear representation⇔
$\iff r(a_1,a_2,\cdots ,a_s)=r(a_1,a_2,\cdots ,a_s,b)$

4. The relationship between the rank of the vector group and the rank of the matrix

设$r(A_{m\times n})=r$ , 则AAThe rank r of$A$$r\left(A\right)$ givenAAThe linear correlation relationship of the row-column vector group of$A\; is:$

(1) 若$r\left(A_{m\times n}\right)=r=m$ , 则$The\; set\; of\; row\; vectors\; of\; A$ is linearly independent.

(2) 若$r(A_{m\times n})=r<m$ , 则$The\; set\; of\; row\; vectors\; of\; A$ is linearly dependent.

(3) 若$r\left(A_{m\times n}\right)=r=n$ , 则AAThe set of column vectors of$A\; is\; linearly\; independent.$

(4) 若$r\left(A_{m\times n}\right)=r<n$ , 则AAThe set of column vectors of$A\; is\; linearly\; related.$

5. $Basis\; Transformation\; Formula\; and\; Transition\; Matrix\; of\; n-$ Dimensional Vector Space

α 1 , α 2 , ⋯ , α n \ alpha_ $a_1,a_2,\cdots ,a_n$与$b_1,b_2,\cdots ,b_n$is the vector space $Two\; sets\; of\; basis\; of\; V$ , then the basis transformation formula is:

$(b_1,b_2,\cdots ,b_n)=(a_1,a_2,\cdots ,a_n)\left[\begin{array}{cccc}{c}_{11}& c_{12}& \cdots & c_{1n}\\ c_{21}& c_{22}& \cdots & c_2\\ \cdots & \cdots & \cdots & \cdots \\ c_n& c_n& \cdots & c_{nn}\end{array}\right]=(a_1,a_2,\cdots ,a_n)C$

where $C$ is an invertible matrix, called by basis$a_1,a_2,\cdots ,a_n$到基$b_1,b_2,\cdots ,b_n$transition matrix.

6. Coordinate transformation formula

If the vector $\gamma$在基$a_1,a_2,\cdots ,a_n$与基$b_1,b_2,\cdots ,b_n$The coordinates are
$X={(x_1,x_2,\cdots ,x_n)}^T$，

$Y={\left(y_1,y_2,\cdots ,y_n\right)}^T$即：$c=x_1{a}_1+x_2{a}_2+\cdots +x_n{a}_n=y_1{b}_1+y_2{b}_2+\cdots +y_n{b}_n$, then the vector coordinate transformation formula is $X=CY$ ψ$Y=C^{-1}X$, where$C$ is from the basis$a_1,a_2,\cdots ,a_n$到基$b_1,b_2,\cdots ,b_n$transition matrix.

7. Inner product of vectors

$(a,b)=a_1{b}_1+a_2{b}_2+\cdots +a_n{b}_n=a^T\beta =b^{T}a$

8. Schmidt Orthogonalization

α 1 , α 2 , ⋯ , α s \ alpha_ $a_1,a_2,\cdots ,a_s$Linearly independent, then $b_1,b_2,\cdots ,b_s$Make it pairwise orthogonal, and $b_i$Only $a_1,a_2,\cdots ,a_i$Linear combination of $(i=1,2,\cdots ,n)$ , recaptured$b_i$Unitization, $c_i=\frac{b_i}{|{\beta }_i|}$,则$c_1,c_2,\cdots ,c_i$is a set of orthonormal vectors. Where
$b_1=a_1$，$b_2=a_2-\frac{(a_2,b_1)}{(b_1,b_1)}{b}_1$，$b_3=a_3-\frac{(a_3,b_1)}{(b_1,b_1)}{b}_1-\frac{(a_3,b_2)}{(b_2,b_2)}{b}_2$ ，

…

$b_s=a_s-\frac{(a_s,b_1)}{(b_1,b_1)}{b}_1-\frac{(a_s,b_2)}{(b_2,b_2)}{b}_2-\cdots -\frac{(a_s,b_{s-1})}{(b_{s-1},b_{s-1})}{b}_{s-1}$

9. Orthogonal bases and orthonormal bases

If the vectors in a set of bases of a vector space are orthogonal to each other, it is called an orthogonal base; if each vector in an orthogonal base is a unit vector, it is called an orthonormal base.

system of linear equations

1. Cramer's Law

线性方程组$\{\begin{array}{l}{a}_{11}{x}_1+a_{12}{x}_2+\cdots +a_{1n}{x}_n=b_1\\ a_{21}{x}_1+a_{22}{x}_2+\cdots +a_2{x}_n=b_2\\ \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \cdots \\ a_n{x}_1+a_n{x}_2+\cdots +a_{nn}{x}_n=b_n\end{array}$, if the coefficient determinant $D=\left|A\right|\ne 0$ , then the system of equations has a unique solution,$x_1=\frac{D_1}{D},x_2=\frac{D_2}{D},\cdots ,x_n=\frac{D_n}{D}$, where $D_j$is the DDjjthin$D$The determinant obtained by replacing the elements in column$j\; with\; the\; constant\; column\; at\; the\; right\; end\; of\; the\; equation\; system.$

2. $n$ -order matrix$A$可逆$\iff Ax=0$ has only zero solutions. $\iff \forall b,Ax=b$ always has a unique solution, in general,$r(A_{m\times n})=n\iff Ax=0$ has only zero solutions.

3. Sufficient and necessary conditions for non-odd order linear equations to have solutions, properties of solutions of linear equations and structure of solutions

(1) Set $A$为$m\times n$ matrix, if$r\left(A_{m\times n}\right)=m$ , then for$Ax=$For$b$$r(A)=r(A⋮b)=m$ , so that$Ax=b$ has a solution.

(2) 设$x_1,x_2,\cdots x_s$For $Ax=The\; solution\; of\; b$ , then$k_1{x}_1+k_2{x}_2\cdots +k_s{x}_s$当$k_1+k_2+\cdots +k_s=1$ is still$Ax=The\; solution\; of\; b$ ; but when$k_1+k_2+\cdots +k_s=0$ , then$Ax=0$ solution. Specifically$\frac{x_1+x_2}{2}$For $Ax=b$ -wise solution;$2x{\_}_3-(x_1+x_2)$ is$Ax=0$ solution.

(3) Inhomogeneous linear equation system $Ax=b$无解$\iff r(A)+1=r(\overline{A})\iff b$ cannot be assigned byAAThe column vector of$A$$a_1,a_2,\cdots ,a_n$Linear representation.

4. Basic solution systems and general solutions of odd linear equations, solution space, general solutions of non-odd linear equations

(1) Homogeneous equation system $Ax=0$ always has a solution (there must be a zero solution). When there are non-zero solutions, since any linear combination of solution vectors is still the solution vector of the homogeneous equation system,$Ax=All\; the\; solution\; vectors\; of\; 0$ constitute a vector space, which is called the solution space of the system of equations, and the dimension of the solution space is$n-r\left(A\right)$ , a set of basis of the solution space is called the basis solution system of the homogeneous equation system.

(2) ${the}_1,{the}_2,\cdots ,{the}_t$is $Ax=$The basic solution system of$0\; ,\; namely:$

1. ${the}_1,{the}_2,\cdots ,{the}_t$is $Ax=0$ solution;

2. ${the}_1,{the}_2,\cdots ,{the}_t$linearly independent;

3. $Ax=$Any solution of$0$${the}_1,{the}_2,\cdots ,{the}_t$线性表出.
   $k_1{the}_1+k_2{the}_2+\cdots +k_t{the}_t$is $Ax=$The general solution of$0$$k_1,k_2,\cdots ,k_t$is an arbitrary constant.

Eigenvalues ​​and Eigenvectors of Matrix

1. The concepts and properties of eigenvalues ​​and eigenvectors of matrices

(1) Let $\lambda$ is$A$ special attack,$kA,aA+bE,A^2,A^m,f(A),A^T,A^{-1},A^{\ast }$ Let be a function of the infinitive
$k\lambda ,al+b,l^2,l^m,f\left(\lambda \right),l,l^{-1},\frac{|A|}{l},$ and the corresponding eigenvectors are the same ($A^T$ exception).

(2)Plant $l_1,l_2,\cdots ,l_n$for $A$ 'snn∑i = 1 n λ i = ∑ i = 1 naii , ∏ i = 1 n λ i = ∣ A ∣ \sum_{i= 1}^{n}\lambda_{i} = \$sum\_$${\sum }_{i=1}^n{l}_i={\sum }_{i=1}^n{a}_{ii},{\prod }_{i=1}^n{l}_i=|A|$ ,denote∣$|A|\ne 0\iff A$ has no eigenvalues.

(3) $l_1,l_2,\cdots ,l_s$for $A$ 's$s$ eigenvalues, the corresponding eigenvectors are$a_1,a_2,\cdots ,a_s$，

Young: $a=k_1{a}_1+k_2{a}_2+\cdots +k_s{a}_s$ ,

则: $A^{n}a=k_1{A}^n{a}_1+k_2{A}^n{a}_2+\cdots +k_s{A}^n{a}_s=k_1{l}_1^n{a}_1+k_2{l}_2^n{a}_2+\cdots k_s{l}_s^n{a}_s$ 。

2. Concept and properties of similarity transformation and similarity matrix

(1) Young $A\sim B$ , then

1. $A^T\sim B^T,A^{-1}\sim B^{-1},,A^{\ast }\sim B^{\ast }$

2. $|A|=|B|,{\sum }_{i=1}^n{A}_{ii}={\sum }_{i=1}^n{b}_{ii},r(A)=r(B)$

3. $|\lambda E-A|=|\lambda E-B|$ , pair$\forall \lambda$ established

3. Sufficient and necessary conditions for matrices to be similarly diagonalizable

(1) Set $A$ is$n$ order square matrix, then$A$ is diagonalizable$\iff$ for each$k_i$Repeated root eigenvalue $l_i$, with $n-r(l_{i}E-A)=k_i$

(2) Set $A$ can be diagonalized, then by$P^{-1}AP=\Lambda ,$有$A=P\Lambda P^{-1}$ ,let$A^n=P{L}^n{P}^{-1}$

(3) Important conclusions

1. 若$A\sim B,C\sim D$，则$\left[\begin{array}{cc}A& O\\ O& C\end{array}\right]\sim \left[\begin{array}{cc}B& O\\ O& D\end{array}\right]$.

2. Young $A\sim B$，则$f(A)\sim f(B),\left|f(A)\right|\sim |f(B)|$ , where$f\left(A\right)$ is about$n$ -order square matrix$A$ polynomial.

3. Young $A$ is a diagonalizable matrix, then the number of its non-zero eigenvalues ​​(repeated calculation of repeated roots) = rank ($A$)

4. Eigenvalues, eigenvectors and similar diagonal matrices of real symmetric matrices

(1) Similarity matrix: Let $A,B$ is two$n$ order square matrix, if there is an invertible matrix$P$ , such that$B=P^{-1}AP$is established, then the matrix$A$ and$B$ is similar, recorded as$A\sim B$。

(2) The nature of the similarity matrix: if $A\sim B$ has:

1. $A^T\sim B^T$

2. $A^{-1}\sim B^{-1}$ (young$A$，$B$ are reversible)

3. $A^k\sim B^k$ （$k$ is a positive integer)

4. $\left|\lambda E-A\right|=|\lambda E-B|$ , so that$A,B$
   have the same eigenvalues

5. $\left|A\right|=|B|$ , so$A,B$ is reversible or irreversible at the same time

6. 秩$\left(A\right)=$秩$\left(B\right),\left|\lambda E-A\right|=|\lambda E-B|$，$A,B$ is not necessarily similar

secondary type

1. $n$个variablex${\mathbf{x}}_1,x_2,\cdots ,{\mathbf{x}}_n$The quadratic homogeneous function of

$f(x_1,x_2,\cdots ,x_n)={\sum }_{i=1}^n{\sum }_{j=1}^n{a}_{ij}{x}_i{y}_j$，其中$a_{ij}=a_{ji}(i,j=1,2,\cdots ,n)$ , called$n$元二次型,简称二次型. Let$x=\left[\begin{array}{c}{x}_1\\ x_1\\ ⋮\\ x_n\end{array}\right],A=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1n}\\ a_{21}& a_{22}& \cdots & a_2\\ \cdots & \cdots & \cdots & \cdots \\ a_n& a_n& \cdots & a_{nn}\end{array}\right]$, this quadratic form $f$ can be rewritten into a matrix-vector form$f=x^{T}Ax$.$\_$Among them$A$ is called a quadratic matrix because$a_{ij}=a_{ji}(i,j=1,2,\cdots ,n)$ , so the quadratic form matrices are all symmetric matrices, and the quadratic form corresponds to the symmetric matrix one by one, and the matrix$The\; rank\; of\; A$ is called the rank of the quadratic form.

2. Inertia theorem, standard form and canonical form of quadratic form

(1) Inertia theorem

For any quadratic form, no matter what kind of contract transformation is selected to convert it into a standard form containing only square terms, its positive and negative inertial indices have nothing to do with the selected transformation, which is the so-called inertia theorem.

(2) Standard form

二次型$f=\left(x_1,x_2,\cdots ,x_n\right)=x^{T}Ax$undergoes contract transformation$x=Cy$化为$f=x^{T}Ax=y^T{C}^{T}AC$

$y={\sum }_{i=1}^r{d}_i{y}_i^2$called $f(r\le n)$ canonical form. In the general number field, the canonical form of the quadratic form is not unique, it is related to the contract transformation, but the number of square terms whose coefficients are not zero is given by r ( A ) r($r\left(A\right)$ is uniquely determined.

(3) Canonical form

Any real quadratic form $f$ can be transformed into canonical form$f=z_1^2+z_2^2+\cdots z_p^2-z_{p+1}^2-\cdots -z_r^2$, where $r$ forAArank of$A$$p$ is the positive inertia index,$r-p$ is a negative inertial index, and the canonical type is unique.

3. Use orthogonal transformation and formula to normalize the quadratic form, and the positive definiteness of the quadratic form and its matrix

AA $A$正定$\Rightarrow kA(k>0),A^T,A^{-1},A^{\ast }$ positive definite;$|A|>0$,$A$ reversible;$a_{ii}>0$，且$|A_{ii}|>0$

$A$，$B$ positive definite$\Rightarrow A+B$ is positive definite, but$AB$，$BA$ is not necessarily positive definite

$A$正定$\iff f(x)=x^{T}Ax>0,\forall x\ne 0$

$\iff$The principal subforms of each order of$A\; are\; all\; greater\; than\; zero$

$\iff$All eigenvalues ​​of$A\; are\; greater\; than\; zero$

$\iff$The positive inertia index of$A$$n$

$\iff$ There is a reversible matrix$P$使$A=P^{T}P$

$\iff$ There is an orthogonal matrix$Q$ ,from$Q^{T}AQ=Q^{-1}AQ\_=\left(\begin{array}{ccc}{l}_1& & \\ \begin{array}{cc}& \\ & \end{array}& \ddots & \\ & & l_n\end{array}\right),$

其中$l_i>0,i=1,2,\cdots ,n.$正定$\Rightarrow kA(k>0),A^T,A^{-1},A^{\ast }$ positive definite;$|A|>0,A$ reversible;$a_{ii}>0$，且$|A_{ii}|>0$ 。

overall framework

Operation nature

reference article

Basic Concepts of Linear Algebra for Machine Learning Mathematical Foundations of Machine Learning (itdiffer.com)

Linear Algebra in Machine Learning - Zhihu (zhihu.com)

Basic Knowledge of Linear Algebra-Mind Map_Linear Algebra Mind Map_Arrow's Blog-CSDN Blog

recommended reading

[Mathematical Basis of Machine Learning] (1) Linear Algebra (Part 1)_linear algebra for everyone csdn_Binary Artificial Intelligence Blog-CSDN Blog

[Mathematical Basis of Machine Learning] (2) Linear Algebra (Medium)_Binary Artificial Intelligence Blog-CSDN Blog

[Mathematical basis of machine learning] (3) Linear Algebra (Linear Algebra) (below)_ordered basis line generation_Binary Artificial Intelligence Blog-CSDN Blog

The most complete knowledge points of linear algebra for the postgraduate entrance examination to sort out the mind map - Zhihu (zhihu.com)

LQLab: Coding Learning Writing — LQLab

This article is published by OpenWrite, a multi-post platform for blogging !