[Numerical calculation method (Huang Mingyou)] Iterative method for solving linear algebraic equations (1): vector, matrix norm and spectral radius [Theory to program]

  Note: For quick reading, you can jump directly to the "4. Summary of Knowledge Points" and "5. Calculation Examples" sections

1. Vector, matrix norm and spectral radius

  When it comes to linear algebra and matrix theory, vectors, matrix norms and spectral radii are very important concepts, which will be detailed below Introducing these:

1. Vector norm

a. Definition and properties

  One piece of thought $n$ 维向量 $x$, define a real-valued function $N\left(x\right)$, Transcription $N(x)=\Vert x\Vert$. Results $N\left(x\right)$ Satisfactory or less condition, nana It is $A\; vector\; norm\; (or\; vector\; module)\; on\; x$:

1. Negativeness: $N\left(x\right)\ge 0$, and $N\left(x\right)=0$to be $x$ is a zero vector.

$$\Vert x\Vert \ge 0$$$$\Vert x\Vert =0\text{this is the case}x=0$$

2. Homogeneity: for any real number $\alpha$(dimensional),if $N(\alpha x)=|\alpha |\cdot N(x)$。

$$\Vert \alpha x\Vert =|\alpha |\cdot \Vert x\Vert$$

3. Triangle inequality: for any vector $x$ sum $y$，有 $N(x+y)\le N\left(x\right)+N(y)$。

   $$\Vert x+y\Vert \le \Vert x\Vert +\Vert y\Vert$$

Additional explanation

• Non-negativity: The norm is non-negative, i.e. it cannot be negative. The norm is zero if and only if the vector is a zero vector.

• Homogeneity: The norm remains consistent under scaling (multiplied by a constant), i.e. scaling up or down a vector affects its norm proportionally.

• Triangle Inequality: The normed triangle inequality means that the distance between two vectors is measured by the sum of both sides. It ensures that the "triangle" in vector space does not become distorted.

norm difference

  It can be deduced from the above triangle inequality: $$\Vert x-y\Vert \ge |\Vert x\Vert -\Vert y\Vert |$$

• The derivation process
  • According to the triangle inequality of vector norm, for any vector $x$ sum $y$，有：$$\Vert x-y\Vert \le \Vert x\Vert +\Vert y\Vert$$ inside

b. Common vector norms

$l_1$、$l_2$、$l_{\infty }$norm

  对于一个 $n$维向量 $x=(x_1,x_2,\dots ,x_n)$ ：

1. $l_1$ 范数：
   $$\Vert x{\Vert }_1=\sum _{i=1}^n|x_i|$$

2. $l_2$ 范数：
   $$\Vert x{\Vert }_2=\sqrt{\sum _{i=1}^n{x}_i^2}$$

3. $l_{\infty }$ 范数：
   $$\Vert x{\Vert }_{\infty }=\underset{1\le i\le n}{\max }|x_i|$$

nature

• Non-negativity:

  $$\Vert x{\Vert }_1,\Vert x{\Vert }_2,\Vert x{\Vert }_{\infty }\ge 0$$

• Nextness： 对于每个 $x$ Summarization quantity $\alpha$, these three norms all satisfy homogeneity, that is,
  $$\Vert \alpha x{\Vert }_1=|\alpha |\cdot \Vert x{\Vert }_1$$ $$\Vert \alpha x{\Vert }_2=|\alpha |\cdot \Vert x{\Vert }_2$$ $$\Vert \alpha x{\Vert }_{\infty }=|\alpha |\cdot \Vert x{\Vert }_{\infty }$$

• Triangle inequality: For each pair of vectors $x$ sum $y$, these three norms all satisfy the triangle inequality:
  $$\Vert x+y{\Vert }_1\le \Vert x{\Vert }_1+\Vert y{\Vert }_1$$$$\Vert x+y{\Vert }_2\le \Vert x{\Vert }_2+\Vert y{\Vert }_2$$$$\Vert x+y{\Vert }_{\infty }\le \Vert x{\Vert }_{\infty }+\Vert y{\Vert }_{\infty }$$

relation

• $l_1$ 范数、$l_2$ 范数、$l_{\infty }$ There is a relationship between norms:
  $$\Vert x{\Vert }_{\infty }\le \Vert x{\Vert }_2\le \sqrt{n}\Vert x{\Vert }_{\infty }$$ $$\Vert x{\Vert }_{\infty }\le \Vert x{\Vert }_1\le n\Vert x{\Vert }_{\infty }$$

2. Matrix norm

a. Norm of matrix

  The norm of a matrix is ​​a real-valued function defined on the matrix space that measures the size or measure of the matrix. For a matrix $A$, the matrix norm is usually expressed as $N\left(A\right)$ or $||A||$, less than or equal to full condition:

1. Non-negativity: For any matrix $A$,if $N(A)\ge 0$, and the equal sign is true if and only if $A$ This is zero square.

2. Homogeneity: For any scalar $k$ 和矩阵 $A$，有 $N(kA)=|k|\cdot N(A)$。

3. Triangle Inequality: For any two matrices $A$ Sum $B$，有 $N(A+B)\le N(A)+N(B)$。

b. Common matrix norms

compatibility norm

• For any two matrices $A$ Sum $B$，有 $||AB||\le ||A||\cdot ||B||$, this is the compatibility of the subject.
• 对于Optional Square $A$ 和向量 $x$，有 $||Ax||\le ||A||\cdot ||x||$, which is also a compatibility property.

Operator norm

Specifically, the commonly used operator norm is $p$number of seats, inside $p$ is a real number.

• tai $p=When\; \infty$, the operator norm is defined as the maximum value of the sum of the absolute values ​​of the matrix rows. That is,
  $$||A|{|}_{\infty }=\underset{1\le i\le n}{\max }\sum _{j=1}^n|a_{ij}|$$
• 当 $p=When\; 1$, the operator norm is defined as the maximum value of the sum of the absolute values ​​of the matrix columns. That is,
  $$||A|{|}_1=\underset{1\le j\le n}{\max }\sum _{i=1}^n|a_{ij}|$$
• 当 $p=2$ time, the number of calculations is determined $Spectral\; radius\; of\; A$. The spectral radius is the modular maximum value of the eigenvalues ​​of the matrix, expressed as $$p(A)=\max |\lambda |$$among $\lambda$ Correct $A$'s special expedition.

3. Spectral radius

  To be improved...

4. Summary of knowledge points

1. Vector norm

• $l_1$ 范范数（蛼哈顿范数）：
  $$||x|{|}_1=\sum _{i=1}^n|x_i|$$

• $l_2$ Number of units (number of units available in Europe):
  $$||x|{|}_2=\sqrt{\sum _{i=1}^n{x}_i^2}$$

• $l_{\infty }$ 范数（无穷范数）：
  $$||x|{|}_{\infty }=\underset{1\le i\le n}{\max }|x_i|$$

2. Matrix norm

• Frobenius norm (the square root of the sum of the squares of each number in the matrix)
  $$||A|{|}_F=\sqrt{\sum _{i=1}^n\sum _{j=1}^n|a_{ij}{|}^2}$$
• Operator norm
  • row sum range : $p=When\; \infty$, the operator norm is defined as the maximum sum obtained by summing the absolute values ​​of the elements in each row of the matrix, that is,
    $$||A|{|}_{\infty }=\underset{1\le i\le n}{\max }\sum _{j=1}^n|a_{ij}|$$
  • Column and norm: when $p=When\; 1$, the operator norm is defined as the maximum value of the sum of absolute values ​​of the
    matrix columns. That is,
    $$||A|{|}_1=\underset{1\le j\le n}{\max }\sum _{i=1}^n|a_{ij}|$$
  • 当 $p=2$ time, calculation number immediately$The\; spectral\; radius\; of\; A$ is the modulus maximum value of the eigenvalues ​​of the matrix
    $$||A|{|}_2=\sqrt{l_{\text{max}}(A^{T}A)}=p(A)=\max |\lambda |$$

3. Spectral radius

  Spectral radius is the value with the largest eigenvalue of the matrix modulo, for a $n\times n$ 的矩阵 $A$, its radius $p\left(A\right)$ Fixed price:

$$p\left(A\right)=\max \{|\lambda ||\lambda \text{This is}A\text{'s special expedition}\}$$

5. Calculation examples

For matrix$$A=\left[\begin{array}{cc}2& 1\\ -1& 4\end{array}\right]$$Calculate its various norms:

∥ A ∥ 1 = max ⁡ j ∑ i ∣ a i j ∣ = max ⁡ { 3 , 5 } = 5 \|A\|_1 = \max_j \sum_i |a_{ij}| = \max\{3, 5\} = 5 ∥A∥1​=jmax​i∑​∣aij​∣=max{ 3,5}=5

$$\Vert A{\Vert }_{\infty }=\underset{i}{\max }\sum _j|a_{ij}|=\max \{3,5\}=5$$

$$\Vert A{\Vert }_2=\sqrt{l_{\text{max}}\left(A^{T}A\right)}$$

calculation $A^{T}A$’s eigenvalue, find the maximum eigenvalue$l_{\text{max}}$：

$$A^{T}A=\left[\begin{array}{cc}5& -2\\ -2& 17\end{array}\right]$$

Special expedition $l_1=11+2\sqrt{10}$, $l_2=11-2\sqrt{10}$。

$$\Vert A{\Vert }_2=\sqrt{l_{\text{max}}}=\sqrt{11+2\sqrt{10}}=4.162277$$

3. Spectral radius:

   p ( A ) = max ⁡ { ∣ λ ∣ } = 3 p(A) = \max \{|\lambda|\} =3p(A)=max{ ∣λ∣}=3

   对$A$ Find the eigenvalues ​​and find the largest absolute value.

• 1 norm: 5
• ∞ norm: 5
• 2 norm: 4.162277
• Spectral radius: 3