The principle of PCA is to project the original sample data into a new space, which is equivalent to mapping a set of matrices to another coordinate system as we learned in matrix analysis. Through a conversion coordinate, it can also be understood as converting one set of coordinates to another set of coordinate systems, but in the new coordinate system, it means that the original does not need so many variables, only the largest linear sample of the original sample is needed. The coordinates of the space corresponding to the eigenvalues of the irrelevant groups are sufficient.
For example, the original sample has a dimension of 30*1,000,000, which means that we have 30 samples, and each sample has 1,000,000 feature points. There are too many feature points, and we need to reduce the dimensionality of the feature points of these samples. Then when reducing the dimension, a covariance matrix of the original sample matrix will be calculated, here is 1000000*1000000, of course, this matrix is too large, there are other ways to process it during calculation, here is just to explain the basic principle, and then through This 1000000*1000000 covariance matrix calculates its eigenvalues and eigenvectors, and finally obtains the eigenvector with the largest eigenvalue to form the transformation matrix. For example, our first 29 eigenvalues can already account for more than 99% of all eigenvalues, then we only need to extract the eigenvectors corresponding to the first 29 eigenvalues. In this way, a transformation matrix of 1000000*29 is formed, and then the original sample is multiplied by this transformation matrix to obtain the corresponding coordinates of the original sample data in the new feature space. 30*1000000 * 1000000*29 = 30 *29, so the number of eigenvalues of each sample of the original training sample is reduced to 29.
Reference article link: http://blog.jobbole.com/109015/
Generally speaking, the dimension of the features of each sample after PCA dimensionality reduction will not exceed the number of training samples, because the excess features are meaningless.