Reprinted: http: //redstonewill.com/1529/
Normal square matrix decomposition (the EVD)
If we know that a matrix A is a square matrix, i.e., the ranks of the same dimension (MXM), A can be generally characterized Decomposition:
Wherein U is a column vector of feature vectors A, Lambda is a diagonal matrix, the diagonal elements Lambda corresponding eigenvalue vectors.
As a simple example, A is for example square:
Then subjected to eigenvalue decomposition, Python code corresponding to:
. 1 Import numpy AS NP 2 . 3 A = np.array ([[2,2 &], [1,2 ]]) . 4 Lamda, the U-np.linalg.eig = (A) # eigenvectors and eigenvalues . 5 Print ( ' matrix A ' , A) . 6 Print ( ' eigenvalues Lamda ' , Lamda) . 7 Print ( ' eigenvectors the U- ' , the U-) . 8 . 9 # output 10 # square matrix A [[2 2] . 11 # [2. 1]] 12 # eigenvalue Lamda [3.41421356 .58578644] 13 is #Eigenvectors the U-[[.81649658 -0.81649658] 14 # [0.57735027 0.57735027]]
A feature is to split decomposition, as follows:
Wherein the eigenvalue λ1 = 3.41421356, corresponding eigenvector u1 = [0.81649658 0.57735027]; eigenvalue λ2 = 0.58578644, corresponding eigenvector u2 = [- 0.81649658 0.57735027], feature vectors are column vectors.
For any matrix, different eigenvectors corresponding to eigenvalues necessarily linearly independent, but not necessarily orthogonal
Symmetric matrix matrix factorization (the EVD)
If the matrix A is a symmetric matrix, for example:
Symmetric matrix decomposition characteristics satisfies the following formula:
Then subjected to eigenvalue decomposition, Python code corresponding to:
. 1 Import numpy AS NP 2 . 3 A = np.array ([[2,1], [1,1 ]]) . 4 Lamda, the U-np.linalg.eig = (A) # eigenvectors and eigenvalues . 5 Print ( ' matrix A ' , A) . 6 Print ( ' eigenvalues Lamda ' , Lamda) . 7 Print ( ' eigenvectors the U- ' , the U-) . 8 . 9 # output 10 # square matrix A [[2. 1] . 11 # [. 1. 1]] 12 # eigenvalue Lamda [2.61803399 .38196601] 13 is #Eigenvectors the U-[[0.85065081 -0.52573111] 14 # [.52573111 0.85065081]]
A feature is to split decomposition, as follows:
Wherein the eigenvalue λ1 = 2.61803399, corresponding eigenvector u1 = [0.85065081 0.52573111]; eigenvalue λ2 = 0.38196601, corresponding eigenvector u2 = [- 0.52573111 0.85065081], feature vectors are column vectors.
Note that, we found that the decomposition and the decomposition of non-symmetric matrix against matrix except for formulas eigenvectors have different characteristics. Symmetric matrix of eigenvectors corresponding to different values in only linearly independent and mutually orthogonal. What is orthogonal it? The inner product is zero is a feature vector. Verify the following:
0.85065081∗−0.52573111+0.52573111∗0.85065081=0
Focus here, after a symmetric matrix A matrix decomposition, can be written as follows:
To verify the formula:
Singular value decomposition (SVD)
We found that, A in the matrix decomposition in a square or symmetric matrix, the ranks of the dimensions are the same. But the actual application, many non-square matrix are asymmetric. So how do you break down on this type of matrix it?
Therefore, we have introduced a method for the decomposition of dimension mxn matrix, known as singular value decomposition (Singular Value Decomposition).
Suppose dimension MXN matrix A, while A is not square, but the following matrix is square, and the dimensions are mxm, nxn
Hence, we can square respectively above decomposition:
Wherein, [Lambda] 1 and Λ2 is the focus matrix, and the diagonal nonzero elements are the same, i.e. have the same square two nonzero eigenvalue, so that the value of σ1, σ2, ..., σk. Notably, k <= m and k <= n
The σ1, σ2, ..., σk matrix A can be obtained eigenvalues
Next, we will be able to get the formula singular value decomposition:
# Formula derivation https://www.cnblogs.com/pinard/p/6251584.html
Wherein, P is called left singular matrix, a dimension mxm, Q is called the right singular matrix, a dimension nxn. Lambda is not square, which dimension mxn, Λ is the diagonal nonzero elements A on the eigenvalues λ1, λ2, ..., λk. Singular value decomposition represented graphically as shown below:
Here is a simple example to illustrate, Let A be a matrix of 3 × 2:
There are:
P calculated eigenvectors and corresponding eigenvalues of σ:
Then, there are:
Q calculated eigenvectors and corresponding eigenvalues of σ:
We can see the characteristic values obtained A:
Finally, the integration of matrix multiplication result, to meet the singular value decomposition formula.
Singular value decomposition and can be written in the form:
Wherein, p1 and q1 are left singular matrix and the eigenvector matrix of right singular.
How to visualize understand SVD
Our singular value decomposition of the picture, and the picture may be written in the form:
In the above formula, λ1, λ2, ..., λk is the descending order.
First of all, if we retain only the largest singular value λ1, rounding other singular values, namely A = λ1p1q1T, then plot, very clear
Increasing the number of singular values front, more and more clear image
Visible, when taking the top 50 largest singular value to reconstruct the image, it has been very clear. We get little different and original image. In other words, as the singular values of selected reconstructed image closer to the original image.
Based on this principle, singular value decomposition can be used for picture compression. For example, in this embodiment, the dimensions of the original picture is 870 × 870, the pixel value is stored a total of: 870 × 870 = 756900. If the number of elements using the SVD, take the first 50 to the maximum singular values, the total memory required is:
(870+1+870)∗50=87050
Obviously, the required storage is greatly reduced. The need to store a lot of high-definition pictures, and limited storage space situation, you can use SVD, singular value to retain the largest number of items, discarding small singular value items can be.
It is worth mentioning that, from largest to smallest singular values decay much faster, in many cases, even 1% of the top 10% of singular value and accounted for more than 99% of all the singular values of the sum. This is a good thing for data compression is. This chart below shows the distribution of the present embodiment singular values and singular values accumulated:
SVD data compression algorithm is illustrated as follows:
Sample Code SVD data compression is:
1 from skimage import io 2 import matplotlib.pyplot as plt 3 from PIL import Image 4 5 img=io.imread('./ng.jpg') 6 m,n = img.shape 7 io.imshow(img) 8 plt.show() 9 10 P, L, Q = np.linalg.svd(img) 11 tmp = np.diag(L) 12 if m < n: 13 d = np.hstack((tmp,np.zeros((m,n-m)))) 14 else: 15 d = np.vstack((tmp,np.zeros((m-n,n)))) 16 17 # k = 50 18 img2 = P[:,:50].dot(d[:50,:50]).dot(Q[:50,:]) 19 io.imshow(np.uint8(img2)) 20 plt.show() 21 22 tmp = np.uint8(img2) 23 im = Image.fromarray(tmp) 24 im.save("out.jpg")
References :
https://zhuanlan.zhihu.com/p/26306568
https://www.zhihu.com/question/22237507/answer/53804902