Why dimensionality reduction and feature selection? ? ?
We know that the ultimate goal of machine learning is to predict, of course, we need to train the data before predicting. Usually we don't use raw data for training, why? Some people may think that the original data contains the most abundant information of the sample, and the raw data without any processing can express the sample most completely. This view is not wrong. However, if we use raw data for direct training, there is a problem that the classifier we designed will get good performance on the training set, but the performance on the test set will become very poor. This is the problem of overfitting. Another problem with direct training with raw data is that the original dimension is too high and takes a long time.
Concept understanding
Feature selection feature selection: also known as variable selection or attribute selection.
It is a way to select a subset of existing attributes for modeling.The main purpose of feature selection is to simplify the model, shorten the training time, avoid the curse of dimensionality, and
enhance the generalization ability of the model.Dimensionality reduction:
By recombining the original features, a new feature is formed, and the principal components are selected.
Commonly used dimensionality reduction methods include PCA and SVD decomposition.The main difference between dimensionality reduction and feature selection is that
the former generates a new feature on the original feature, while the latter just selects a subset of the original feature set without modifying the original set.
Feature Selection methods
- Feature Selection – Wrapper method
- Feature Selection – Filter method
- Approaches for Feature Selection
- Summary: the modern approaches for Feature Selection
