Detailed interpretation of vector and matrix norms

1. Vector

• L0 norm: the number of non-zero elements in the vector, also called 0 norm
• L1 norm: It is the sum of absolute values, also called the norm or 1 norm.
• L2 norm: module in the usual sense, also called 2 norm
• p norm: that is, pp of the absolute value of the vector element1 / p 1/pof the sum of$p\; powers$$1/p$ power
• $\infty$ norm: take the maximum value of the vector
• $-\infty$ norm: take the minimum value of the vector

Suppose there is a vector $x=(x_1,x_2,\dots ,x_n{)}^T$ , the norm of the vector is:
$$||x|{|}_1=|x_1|+|x_2|+\cdots +|x_n|$$

$$||x|{|}_2=(|x_1|+|x_2|+\cdots +|x_n|{)}^{\frac{1}{2}}$$

$$||x|{|}_p=(|x_1|+|x_2|+\cdots +|x_n|{)}^{\frac{1}{p}}$$

∣ ∣ x ∣ ∣ ∞ = max ⁡ { ∣ x 1 ∣ + ∣ x 2 ∣ + ⋯ + ∣ x n ∣ } ||x||_{\infty} = \max\{|x_1| + |x_2| + \dots + |x_n|\} ∣∣x∣∣∞​=max{ ∣x1​∣+∣x2​∣+⋯+∣xn​∣}

∣ ∣ x ∣ ∣ − ∞ = min ⁡ { ∣ x 1 ∣ + ∣ x 2 ∣ + ⋯ + ∣ x n ∣ } ||x||_{-\infty} = \min\{|x_1| + |x_2| + \dots + |x_n|\} ∣∣x∣∣−∞​=min{ ∣x1​∣+∣x2​∣+⋯+∣xn​∣}

2. Matrix

There are mainly three types of matrix norms. Remember, the three types of norms are different! !

2.1. Induced norm

Also called operator norm, it is defined that matrix A is located in the matrix space ${\mathbb{R}}^{m\times n}$ 上
∣ ∣ x ∣ ∣ = max ⁡ { ∣ ∣ A x ∣ ∣ ; x ∈ R n , ∣ ∣ x ∣ ∣ = 1 } = m a x { ∣ ∣ A x ∣ ∣ ∣ ∣ x ∣ ∣ , x ∈ R n , x ≠ 0 } ||x||= \max\{||Ax||;x\in \mathbb{R}^n,||x||=1 \} = max\left\{ \frac{||Ax||}{||x||}, x\in \mathbb{R}^n,x\ne 0 \right\} ∣∣x∣∣=max{ ∣∣Ax∣∣;x∈Rn,∣∣x∣∣=1}=max{ ∣∣x∣∣∣∣Ax∣∣​,x∈Rn,x=0 }
The commonly used induced norm is the p norm, also called$L_p$ 范数
$$||A|{|}_p=max\left\{\frac{||Ax|{|}_p}{||x|{|}_p},x\ne 0\right\}$$
0 norm: the number of non-zero elements in the matrix.

1 norm is also called column norm
$$||A|{|}_1=\max \sum _{i=1}^m|a_{ij}|,1\le j\le The\; n-$$
infinity norm is also called the row norm
$$||A|{|}_{\infty }=\max \sum _{j=1}^n|a_{ij}|,1\le i\le The\; n$$
2 norm is also called the spectral norm. The spectral norm of the matrix is ​​the maximum singular value of the matrix
$$||A|{|}_{spec}=||A|{|}_2=p_{max}(A)=\sqrt{l_{max}(A^{T}A)}$$
$p_{max}$It means finding the maximum singular value of the matrix, spec means spectrum.

2.2. Element norm

Let $m\times The\; n$ matrix is ​​first arranged into a$mn\times 1$ vector, and then use the norm definition of the vector, that is, the norm of the matrix
$$||A|{|}_P={\left(\sum _{i=1}^m\sum _{j=1}^n|a_{ij}{|}^p\right)}^{\frac{1}{p}}$$
1 norm is also called sum norm or L1 norm
$$||A|{|}_1=\sum _{i=1}^m\sum _{j=1}^n|a_{ij}|$$
2 norm is also called Forbenius norm∣
$$||A|{|}_F=||A|{|}_2={\left(\sum _{i=1}^m\sum _{j=1}^n|a_{ij}{|}^2\right)}^{\frac{1}{2}}=\sqrt{tr(A^{H}A)}$$
$\infty$ norm is also called the maximum norm
∣ ∣ A ∣ ∣ p = max ⁡ { ∣ aij ∣ } ||A||_p = \max\{ |a_{ij}| \}∣∣A∣∣p​=max{ ∣aij​∣}

2.3. Schatten norm

Using the norm defined by the singular values ​​of the matrix, let the singular values ​​of the matrix form a vector $p=\left[\begin{array}{cccc}{p}_1& p_2& \dots & p_k\end{array}\right]$ , $k=\min (m,n)$

p
$$||A|{|}_p=||\sigma |{|}_p=(\sum _{i=1}^k{p}_i^p{)}^{\frac{1}{p}}$$
p=1, also called the nuclear norm, is the sum of the singular values ​​of the matrix
$$||A|{|}_1=\sum _{i=1}^k{p}_i=tr(\sqrt{A^{H}A})$$
p=2, Schattern norm is equivalent to Frobenius

$$||A|{|}_1=(\sum _{i=1}^k{p}_i^2{)}^{\frac{1}{2}}=\sqrt{tr(A^{H}A)}={\left(\sum _{i=1}^m\sum _{j=1}^n|a_{ij}{|}^2\right)}^{\frac{1}{2}}$$

p=$\infty$ , the Schattern norm is the same as the spectral norm

$$||A|{|}_{\infty }=p_{max}(A)$$