Talking about feature scaling in machine learning

introduction

When using some machine learning algorithms, it is inevitable to perform feature scaling on the data. For example, in the stochastic gradient descent algorithm, feature scaling can sometimes improve the convergence speed of the algorithm. Below I will mainly introduce some methods of feature scaling.

What is feature scaling

Feature scaling is used to normalize the range of data features.

Why Machine Algorithms Need Feature Scaling

Feature scaling can also make machine learning algorithms work better. For example, in the K-nearest neighbor algorithm, the classifier mainly calculates the Euclidean distance between two points. If a feature has a larger range value than other features, the distance will be dominated by this feature value. Therefore, each feature should be normalized, such as the range of values ​​from 0 to 1.

As I said in the introduction, feature scaling can also speed up gradient convergence.

Some methods of feature scaling

Rescaling

This approach scales the features of the data to be between [0,1] or [-1,1]. What extent to zoom to depends on the nature of the data. The formula for this method is as follows:

x′=x−min(x)max(x)−min(x)

xis the original eigenvalue,x′is the scaled value.

Standardization

Feature normalization makes the value of each feature have zero-mean (zero-mean) and unit-variance (unit-variance). This method is widely used in machine learning algorithms. For example: SVM, logistic regression and neural network. The formula for this method is as follows:

x′=x−x¯σ