A linear regression model and least squares

table of Contents

preface

Supervised learning, the prediction is discrete variables, continuous variables called classification otherwise known as regression.

Return contains only one independent variable and the dependent variable, both linear relationship, for the simple linear model.

If the argument is greater than equal to two, for the poly (linear model).

A linear regression model

The overall regression of Y to X function but linear, but also non-linear. Of "linear" linear regression model, there are two explanations:
(1) in terms of linear variables, conditional mean Y is a linear function of X
(2) in terms of the parameters is linear, Y is a parameter conditional mean linear function of the
linear regression model mainly refers to the terms of the parameter is "linear", because for as long as the parameter is linear, its parameters can be estimated in a similar way.

Parameter estimation (least squares)

n points in the plane, find a simple linear function fitting them, so that (the minimum residual total) Total fitting error, three criteria can be selected:

(1) "and the minimum residual" is a way to determine the linear position. But soon found that computing "residuals and" cancel each other out there is a problem.
(2) "and the absolute value of the minimum residual" is a way to determine the linear position. But calculates the absolute value of too much trouble.
Principles of least squares method (3) is "the minimum residual sum of squares" determining the linear position. In addition to calculating the least square method is convenient, the estimate obtained also has excellent properties. This method is very sensitive to outliers.

Least square method (Least Square) so that the residual square sum is minimized.

Sample regression model

$\LARGE Y_i=\hat{\beta_1}+\hat{\beta_2} X_i+e_i$

which is $\LARGE e_i=Y_i-\hat{\beta_1}-\hat{\beta_2} X_i$

Residual sum of squares function Q

$\LARGE \boldsymbol{Q}=\sum_{i=1}^{n} e_i^2=\sum_{i=1}^{n} \left (Y_i-\hat{Y}_i \right )^2=\sum_{i=1}^{n} \left (Y_i-\hat{\beta_1}-\hat{\beta_2X_i}\right )^2$

To make Q minimum, the need for Q derivative, Q partial derivative of the two parameters to be estimated as

$\LARGE \frac{\partial Q}{\partial \beta_1}=2\sum_{i=1}^{n}(Y_i-\hat{\beta_1}-\hat{\beta_2 }X_i)(-1)=0$

$\LARGE \frac{\partial Q}{\partial \beta_2}=2\sum_{i=1}^{n}(Y_i-\hat{\beta_1}-\hat{\beta_2 }X_i)(-X_i)=0$

Solutions have to

$\LARGE \hat{\beta }_1 = \frac{\sum X_i^2 \sum Y_i-\sum X_i\sum X_iY_i }{n\sum X_i^2- (\sum X_i)^2}$

$\LARGE \hat{\beta }_2 = \frac{n\sum X_iY_i-\sum X_i\sum Y_i }{n\sum X_i^2- (\sum X_i)^2}$

C ++ realize the least squares method

#include<iostream>
#include<fstream>
#include<vector>

using namespace std;

class LeastSquare{

double a,b;
public:
        LeastSquare(const vector<double> &x,const vector<double>& y){
            double t1=0,t2=0,t3=0,t4=0;
            for(int i=0;i<x.size();++i)
            {
    
			    t1 += x[i]*x[i];

			    t2 += x[i];

			    t3 += x[i]*y[i];

			    t4 += y[i];
            }

 	    	a = (t3*x.size() - t2*t4) / (t1*x.size() - t2*t2);
		    b = (t1*t4 - t2*t3) / (t1*x.size() - t2*t2);
        }
    
        double getY(const double x) const
        {
            return a*x+b;
        }


        void print() const
        {

	    	cout<<"y = "<<a<<"x + "<<b<<"\n";

	    }

};



int main(int argc,char *argv[])
{
    vector<double> x;
    ifstream in(argv[1]);
    for(double d; in>>d)
            x.push_back(d);
    int len=x.size();

    vector<double> y(x.begin()+len/2,x.end());
    x.resize(len/2);
    LeastSquare ls(x,y);
    ls.print();
    
    cout<<"Input x=:\n";
    double x0;
    while(cin>>x0)
        {
            cout<<"y = "<<ls.getY(x0)<<endl;
            cout<<"Input x:\n";
        }
    return 0;
}

Bluenapa

发布了128 篇原创文章 · 获赞 21 · 访问量 3万+

私信关注

A linear regression model and least squares

preface

A linear regression model

Parameter estimation (least squares)

C ++ realize the least squares method

Guess you like