EDITORIAL 0
It represents a dimension reduction is called the original high-dimensional feature vectors in a low dimensional. Common dimension reduction methods are principal component analysis, linear discriminant analysis, isometry, locally linear embedding, Laplacian feature mapping, Locality Preserving Projection .
01 PCA theoretical maximum variance
Q1: how to define the main ingredient? From this definition, how to design such that the objective function to extract the main purpose of reducing Vader components? For this objective function, how to solve the problem of the PCA?
A1: PCA aimed at finding the principal component data, and use these principal components characterizing the raw data, so as to achieve the purpose of dimensionality reduction. See proof process P75-77
02 PCA least square error theory
Q1: PCA solved in fact the best projection direction, that is a straight line, which is a mathematical problem in linear regression goals coincide, whether the definition of PCA from the perspective of return objectives and accordingly solve the problem?
A1: a linear regression problem is solved so that the linear function corresponding to a straight line to better fit the sample set. If the definition of the target PCA from this perspective, then the problem will be transformed into a regression problem. Proof see P79-81
03 linear discriminant analysis
LDA is a machine learning, data mining based popular classic in the field. Compared to PCA, LDA can be used as a kind of dimensionality reduction algorithm supervision. In the PCA, the algorithm does not consider the tag data (category), but the original data is mapped onto some of the larger variance direction only.
Q1: For data with a class label, it should be how to design objective function makes the process of dimensionality reduction without loss of information category? In this goal, it should be how to solve?
A1: LDA for classification is first service, so long as a projection direction to find w, such that the sample in the original projection as a separate category.
---- between the central idea of maximizing the LDA-based distance and the minimum distance classes. Because the model is too simple, expressive power has some limitations, we can expand through the introduction of kernel LDA method to handle the distribution of more complex data.
04 linear discriminant analysis in the principal component analysis
Q1: LDA and PCA as a classic of dimensionality reduction algorithm, how to analyze the similarities and differences of principle from the application point of view? Two dimensionality reduction algorithm What is the difference with the contact on the objective function is derived from a mathematical point of view?
A1: starting from the target, PCA is the largest selection of the direction of the projection data variance, and LDA selection is large variance between classes within the subcategories variance projection direction.
From the application point of view, can grasp a basic principle ----- use PCA to reduce the dimension of the task unsupervised, supervised the application of the LDA.