Anomaly detection study notes three, linear regression method, principal component analysis, support vector machine

1. Linear regression method

        Functions like this are the basis of linear regression models and support vector machines. Linear functions are simple. If the original problem is nonlinear, it is easier to convert it into a linear problem, such as the following equation.

         Linear maps are an important part of principal component analysis.

        Look for a linear relationship between the response (dependent variable) and the explanatory variable (independent variable), such as the chart below, which is about test scores and study time, a set of normal data, and a data with outliers. The vertical distance of the line fit is used to calculate the score.

        Ideally, split the data into training and test datasets, use the training data to obtain the parameters for a line fit, and determine a distance threshold for anomalies, then apply the results to the test data to detect anomalies.

         For simple linear regression, assume:

        Goal: Find estimates of the fitted parameters