[Data preprocessing] Data dimensionality reduction and feature extraction

Data dimensionality reduction and feature extraction are common techniques used in data preprocessing. They aim to reduce the dimensionality of data to improve model performance or reduce computational complexity. However, their goals and approaches differ slightly:

1. Data dimensionality reduction :

   • Goal : The purpose of data dimensionality reduction is to reduce the feature dimensions of the data while retaining as much original information as possible so that the data can be processed and analyzed more efficiently.

   • Methods : Commonly used data dimensionality reduction methods include principal component analysis (PCA), linear discriminant analysis (LDA), etc. PCA attempts to find the most dominant directions (principal components) in the data to represent the data, thereby projecting the data into a low-dimensional space. LDA is a supervised learning dimensionality reduction method that takes category information into account and maps the data to a low-dimensional space that can best distinguish different categories.

   • Applicable scenarios : Data dimensionality reduction is suitable for when the data has high dimensions but redundant information. It can help reduce the cost of computing resources, improve the training efficiency of the model, and reduce the risk of over-fitting of the model.

2. Feature Extraction :

   • Goal : Feature extraction is to find more discriminating features by transforming the original data into a new feature space for modeling and prediction.

   • Method : Commonly used feature extraction methods includeBased on statisticsmethods (such as mean, variance, correlation coefficient, etc.),Based on frequency domainmethods (such as Fourier transform, etc.),Based on information theorymethods (such as mutual information, information gain, etc.), etc.

   • Applicable scenarios : Feature extraction is usually used when the original features contain a lot of noise or redundant information, and it is hoped to filter out features that are more useful for the target task.

Summary of differences :

• The purpose of data dimensionality reduction is to reduce the dimensions of data to reduce computational complexity or facilitate visualization while retaining as much information as possible.
• The purpose of feature extraction is to extract features that are more meaningful or discriminative for the task from the original features to improve the performance of the model.

Both techniques are often used together in real-world tasks to better prepare data for modeling and analysis.

Classic data preprocessing process