[Ch05-00] multivariable linear regression problem

Series blog, the original author maintained on GitHub: https://aka.ms/beginnerAI ,
click on the star with a star do not mean more stars the author harder.

Chapter 5 of the single multiple-input single-output neural network

More than 5.0 Linear Regression issues

5.0.1 Ask a question

Question: Beijing Tongzhou, Tongzhou District, 15 km from the center of a 93-square-meter house, about how much money?

Rates prediction problem, has become a topic of machine learning entry of the famous Boston housing data and related competitions already a lot, but the US is the single-family house, garage forecourt backyard swimming pool and so many parameters, early scholars may be difficult to understand. We may wish to use a simplified version of Beijing Tongzhou prices to, for example, feel the process of forecasting prices.

Factors affecting Beijing Tongzhou prices are many, living area, geographic location, orientation, school district house, the surrounding facilities, buildings and so on year, of which, area and location are two of the more important factors. Latitude and longitude location information commonly used way of representation, but the latitude and longitude are two characteristic values, together makes sense, therefore, we convert it to become a center from Tongzhou District.

We have 1000 samples, wherein each sample has two values, a value tag, the example as shown in Table 5-1.

Table 5-1 sample data

Sample number	Location	Living area	Price (million)
1	10.06	60	302.86
2	15.47	74	393.04
3	18.66	46	270.67
4	5.20	77	450.59
...	...	...	...

• Characteristic value 1 - Location, statistics obtained:
  • Max: 21.96 km
  • Minimum: 2.02 km
  • Average: 12.13 km
• Eigenvalues ​​2 - housing area, get statistics:
  • Maximum: 119 square meters
  • Minimum: 40 square meters
  • Average: 78.9 meters
• Tag value - prices, in millions:
  • Max: 674.37
  • Minimum: 181.38
  • Average: 420.64

This data is three-dimensional, can be used as the two feature values ​​of x and y, as the label value Z, as shown in Table 5-2 demonstrate in xyz coordinates.

Table 5-2 Sample visualization of three-dimensional space

Forward	Lateral

From the positive perspective, much like a lawn, it seems to be a plane. Again viewed from the side, and Chapter 4, like a straight line fitted to the data. Therefore, for such three-dimensional linear fit, we can think about it as a fitting plane, this plane will be located in this "lawn" of bits, the "lawn" is divided into two upper and lower thinner "lawn" , so that eventually all the sample points and the minimum squared distance of this plane.

5.0.2 Multiple linear regression model

As the table may not be exactly in line with data 15 kilometers, 93 square meters of conditions, so we need to build a model based on 1000 sample values, to solve the prediction problem.

By way of illustration, we can basically identify the problem is a problem of linear regression, and is typical of multiple linear regression, regression i.e. comprising two or more independent variables. Multivariate linear regression function model as follows:

\[y=a_0+a_1x_1+a_2x_2+\dots+a_kx_k\]

Specific rates of the forecasting problems, the above formula can be simplified to:

\[ z = x_1 \cdot w_1 + x_2 \cdot w_2 + b \]

When the present embodiment RATE problems aside, for general application problems, multiple linear regression model, the regression model in order to ensure an excellent ability to explain and predict the effects, should first be selected independent variables, which criteria are:

1. The argument must have a significant effect on the dependent variable, and showed a close linear correlation;
2. Linear correlation between independent variables and the dependent variable must be real, not formal;
3. From between mutually exclusive variables should have certain properties, i.e. the correlation between independent variables should not exceed the argument relevance because of the dependent variable;
4. Arguments should have complete statistical data, it is easy to determine the predictive value.

5.0.3 Solution

If you use traditional mathematical methods to solve this problem, we can use the normal equation, mathematical analytical solution thus obtained, and then using a neural network approach to obtain approximate solutions, so the accuracy of comparison between the two, further debugging parameters of the neural network, to achieve the purpose of learning.

We might as well put in two ways here to do a comparison, after the reader and run the code, get results, then come back here to experience carefully compare items in Table 5-3.

Table 5-3 Comparison of two methods

method	The normal equation	Gradient descent
principle	Several matrix operations	Multiple iterations
special requirements	\ (X ^ TX \) the inverse exists	You need to determine the learning rate
the complexity	\ (O (n ^ 3) \)	\ (O (n ^ 2) \)
Applicable number of samples	\(m \lt 10000\)	\(m \ge 10000\)