1 scalars, vectors, matrices, and tensors
2 Matrix and vector multiplication
3 Identity and inverse matrices
3.0 Identity Matrix
a = np.identity(3) # 三行三列单位矩阵3.1 Inverse of a matrix
A = [[1.0,2.0],[3.0,4.0]]
A_inv = np.linalg.inv(A)
print("A 的逆矩阵", A_inv)3.1 Transpose
A = np.array([[1.0,2.0],[1.0,0.0],[2.0,3.0]])
A_t = A.transpose()
print("A:", A)
print("A 的转置:", A_t)3.2 Matrix addition
a = np.array([[1.0,2.0],[3.0,4.0]])
b = np.array([[6.0,7.0],[8.0,9.0]])
print("矩阵相加： ", a + b)3.3 Matrix Multiplication
m1 = np.array([[1.0,3.0],[1.0,0.0]])
m2 = np.array([[1.0,2.0],[5.0,0.0]])
print("按矩阵乘法规则： ", np.dot(m1, m2))
print("按逐元素相乘： ", np.multiply(m1, m2))
print("按逐元素相乘： ", m1*m2)
v1 = np.array([1.0,2.0])
v2 = np.array([4.0,5.0])
print("向量内积： ", np.dot(v1, v2))4 Linear correlation and generative subspaces
5 norm
5.0 Code Implementation
5.0.1 L1 L2 infinity norm of a vector
a = np.array([1.0,3.0])
print("向量 2 范数", np.linalg.norm(a,ord=2))
print("向量 1 范数", np.linalg.norm(a,ord=1))
print("向量无穷范数", np.linalg.norm(a,ord=np.inf))5.0.2 F-norm of matrices
a = np.array([[1.0,3.0],[2.0,1.0]])
print("矩阵 F 范数", np.linalg.norm(a,ord="fro"))5.1 General definition of norm
A norm is a function that maps a vector to non-negative values. Intuitively, the norm of a vector x measures the distance from the origin to point x.
5.1.1 Definitions
5.1 L1 norm
5.1.1 Definitions
When the difference between zero and non-zero elements is important in machine learning problems, and in these cases the slope is the same at each position instead, the L1 norm is often used, also often as a surrogate function for the number of non-zero elements .
5.1.2 Mathematical expressions
5.2 L2 norm/Euclidean norm
represents the Euclidean distance from the origin to the point determined by the vector x.
5.2.1 Squared L2 norm
Also often used to measure the size of a vector, which can be calculated simply by a dot product, which grows very slowly near the origin.
5.2.2 Comparison of squared L2 norm and L2 norm
The derivative of the squared L2 norm with respect to each element in x depends only on the corresponding element.
The derivative of the L2 norm with respect to each element is related to the entire vector.
5.3 Maximum norm
5.3.1 Meaning
Represent the absolute value of the element with the largest magnitude in a vector:
5.3.2 Definitions
5.4 F-norm
5.4.1 Meaning
F-norm is used to measure the size of the matrix
5.4.2 Mathematical definitions
5.5 Using the norm to represent the dot product
6 Special Matrix
6.1 Diagonal matrices
Contains non-zero elements only on the main diagonal, all other positions are zero.
6.1.1 Meaning of Diagonal Matrix
By restricting some matrices to be diagonal, we can get computationally inexpensive (and concise) algorithms.
6.1.2 Multiplication of Nonstandard Diagonal Matrices
6.2 Symmetric Matrices
6.3 Orthogonal matrices
Row and column vectors are respectively standard orthogonal square matrices:
6.4 Unit Vectors and Orthogonal
6.5 Standard Orthogonal
If these vectors are not only orthogonal to each other, but also have a norm of 1, then we call them standard orthogonal.
7 Eigen decomposition
7.0 Code Implementation: Feature Decomposition
A = np.array([[1.0,2.0,3.0],
[4.0,5.0,6.0],
[7.0,8.0,9.0]])
# 计算特征值
print("特征值:", np.linalg.eigvals(A))
# 计算特征值和特征向量
eigvals,eigvectors = np.linalg.eig(A)
print("特征值:", eigvals)
print("特征向量:", eigvectors)7.1 Definitions
Decompose a matrix into a set of eigenvectors and eigenvalues.
7.2 Calculation method
7.3 Tip
Not every matrix can be decomposed into eigenvalues and eigenvectors.
Every real symmetric matrix can be decomposed into real eigenvectors and real eigenvalues.
7.4 Positive definite, positive semi-definite, negative definite, semi-negative definite matrices
- A matrix in which all eigenvalues are positive is called a positive definite matrix.
- A matrix in which all eigenvalues are non-negative is called a positive semi-definite matrix.
- A matrix in which all eigenvalues are negative is called a negative definite matrix.
- A matrix whose eigenvalues are all non-positive numbers is called a semi-negative definite matrix.
8 Singular value decomposition (SVD)
8.0 Code to implement singular value decomposition
A = np.array([[1.0,2.0,3.0],
[4.0,5.0,6.0]])
U,D,V = np.linalg.svd(A)
print("U:", U)
print("D:", D)
print("V:", V)8.1 Meaning
Decompose a matrix into singular vectors and singular values.
8.1 Decomposition calculation method
9 Pseudo-inverse
9.1 Problems solved
If the number of rows of matrix A is greater than the number of columns, then the above equation may not have a solution. If the number of rows of matrix A is less than the number of columns, then the above matrix may have multiple solutions.
9.2 Calculation process
10 trace operations
10.1 Definition of trace
10.2 The trace operation provides a way to describe the Frobenius norm of a matrix:
10.3 Operation rules
10.3.1 The trace operation is invariant under the transpose operation
10.3.2 The trace of a square matrix obtained by multiplying multiple matrices is the same as the trace of multiplying by moving the last of these matrices to the front.
10.3.3 After the cyclic permutation, the shape of the matrix obtained by the matrix product changes, and the result of the trace operation remains unchanged.
10.3.4 A scalar remains itself after a trace operation
11 Determinant
11.1 Definitions
The determinant, denoted det(A), is a function that maps a square matrix A to real numbers.
11.2 Characteristics of Determinants
- The determinant is equal to the product of the matrix eigenvalues.
- The absolute value of the determinant can be used to measure how much the space expands or shrinks after the matrix participates in matrix multiplication.
- If the determinant is 0, then the space is completely contracted along at least one dimension, causing it to lose all volume.
- If the determinant is 1, then this transformation keeps the spatial volume unchanged.