Deep Learning reading notes (a)

1. What is called the matrix decomposition?

To satisfy Av = λv, matrix decomposition is divided into: a singular value decomposition and eigenvalue decomposition

Decomposition Characteristics: confined to the square sign, Av = λv where v is the eigenvector matrix of A, if the equations are valid, then λ is the eigenvalue corresponding to each matrix containing only unique eigenvalue and the corresponding eigenvector.

Singular value decomposition: any matrix used in personalized recommendation, the PCA dimension reduction and the NLP, UΣV formula = A ^T , which $The$ is a $m\times m$ matrix $\Sigma$ is a $m\times n$ matrix, in addition to the elements on the main diagonal are all 0 each element on the main diagonal singular values are referred to, $V$ it is a $n\times n$ matrix.

How to solve the U, Σ, V three matrices

If we transpose and A A do matrix multiplication, it will be a square matrix of n × n $A^{T}A$ . Since it $A^{T}A$ is a square, then we can feature decomposition, resulting eigenvalues and eigenvectors satisfy the following formula:

So that we can obtain a matrix $A^{T}A$ of n eigenvalues and corresponding eigenvectors v of n. The $A^{T}A$ all eigenvectors into an n × n matrix V, which is of the formula V we SVD of the matrix. Generally, we will each feature vector V is called the right singular vectors of A.

If we transpose of A and A do matrix multiplication, it will be a square matrix of m × m $AA^{T}$ . Since it $AA^{T}$ is a square, then we can feature decomposition, resulting eigenvalues and eigenvectors satisfy the following formula:

So that we can obtain a matrix $AA^{T}$ of m corresponding to the m eigenvalues and eigenvectors of u. The $AA^{T}$ all eigenvectors into an m × m matrix U, which is our SVD U-matrix of the equation. Generally, we will each feature vector U of A is called the left singular vectors.

U and V we find out, now left singular value matrix Σ not find the.

Since Σ is in addition to other locations singular values on the diagonal are 0, then we need a requirement for each singular value σ on it.

We have noticed:

So we can find each of our singular value, and then find the singular value matrix Σ.

Above there is a problem is not speaking, we say that $A^{T}A$ the feature vector is composed of our SVD matrix V, and

$AA^{T}$ The feature vector is composed of U in our SVD matrix, what according to you? This is actually very easy to prove, we have to prove V matrix as an example.

The type of proof of use ${U ^ U = I}, \ Sigma ^ {T} = \ Sigma$ . As can be seen $A^{T}A$ in the feature vector V it is indeed composed of a matrix of our SVD. Similar methods can be obtained $AA^{T}$ feature vector is composed of U matrix of our SVD.

Further, we can also see that our eigenvalues of matrix singular value is equal to the square of the matrix, that is to say the eigenvalues and singular values satisfy the following relationship:

This means that we can not $\ Sigma_ {i} = \ frac {Av_ {i}} {u_ {i}}$ be calculated singular values can be determined by $A^{T}A$ the value of the square root of the features to find singular values.

SVD algorithm commonly used in the recommendation, data reduction and noise reduction, parallelized

Deep Learning reading notes (a)

Guess you like