Transfer: https: //blog.csdn.net/ccnt_2012/article/details/81127117
Principles and solving relatively simple, as the record about their own learning
1 daily and I do not know
Look at an example in life. For example, there are five ruler:
They are used to measure the length of the line segment, values are obtained (refer to a different color ruler):
The reason why different values may be because:
Different manufacturers produce different precision ruler
Ruler different materials, different thermal expansion and contraction
When measured mood ups and downs
......
Anyway, there is an error, in this case, as a general average value segment length:
Daily life is so used. But as the very thing 'er lover of mathematics, the natural order under:
It makes sense to do so?
With the harmonic mean okay?
Okay with median?
With a geometric mean okay?
2 least-squares method
To consider the issue just put it another thought.
First, the value obtained by the test Videos in a Cartesian coordinate system, are designated as:
Secondly, to guess the true value of the length of a line with a straight line parallel to the horizontal axis represents (because it is guessed, to draw it by a broken line), referred to as:
Each point is made perpendicular to the vertical line of length is, as may be appreciated that an error between the measured value and the true value:
Because the error is the length, but also the absolute value of arithmetic trouble, simply represented squared error:
The total square error is:
Because it is speculation, so be changing:
Naturally, the total error is constantly changing.
French mathematician, Adrien - Marie Legendre (1752-1833, the picture a bit abstract) proposed to let squared total error is the smallest true value, which is based, if the error is random, it should be around the true value fluctuate (this may have a look on "how to understand unbiased estimate?").
This is the least squares method, namely:
This conjecture is also quite intuitive to count.
This is a quadratic function to obtain the minimum value of its time derivative, the derivative is zero:
and then:
It happens to be the arithmetic mean.
The original arithmetic average of the smallest error can make, ah, it seems the choice it appears under the reason.
This method is the following:
Is the least squares method, the so-called "two-by" is the meaning of the square, Taiwan translated directly into the least squares method.
3 promotion
The arithmetic mean of the least squares method is just a special case, the scope is rather narrow. The least squares method use on a wide range.
Such as temperature and ice cream sales:
Looks like some kind of a linear relationship:
This linear relationship can be assumed:
Thought of by the least squares method:
Figure were:
The total squared error is:
Different lead to different, based on knowledge of multivariate calculus, and when:
This time taking the minimum.
For the equations of linear equations, solved the problem before data:
This is the straight lines:
In fact, it is also assumed that:
Under this assumption, according to the least square method, it is calculated to obtain a quadratic curve below this root red:
Same set of data, different options can be different fit curves (source) by the least square method:
Different data, but also to select a different, not the same as can be obtained by least squares curve fit:
Can not choose an arbitrary function, there are some stress, not presented here.
4 with the least squares normal
We guess Legendre, that is the least squares method, still skeptical, if this guess is wrong how to do?
Prince of Mathematics Gauss (1777-1855) like us skeptical.
Gauss a change of thinking framework to think through that set of probability and statistics.
Let us return to the issue of the length of the initial measurement of the line. Gaussian thought, by the measured values:
Each of the measured values and the segment length between the true value of a presence of an error:
These errors eventually form a probability distribution, just do not know the error probability distribution now is. Assume that the probability density function is:
Further assume that a joint probability density function, so that the convenience of all the measured data utilized:
Talking about this, some students may have seen the above likelihood function of (regarding likelihood function and maximum likelihood estimation want to address right away, you can refer to "How to understand the maximum likelihood estimation?").
Because it is about function, and is also a probability density function (distribution graph below is just painting):
According to the idea of maximum likelihood estimation, the probability of the biggest most should appear (Since they appeared, and I'm not a "Chosen only", then naturally not a small probability event occurs), that should be taken to the following this point:
When the following formula was established to obtain the maximum value:
Then Gauss think the answer given by the least squares method is:
If the least squares method is right, then when should obtain the maximum value, namely:
Well, now may be the solution to this differential equation. Finally get:
what is this? This is normal ah.
And this is a necessary and sufficient condition:
That is, if the distribution of the errors are normally distributed, then you will get the least squares method is the most likely value.
Then the distribution of the errors are normally distributed it?
We believe that is a result of random error, numerous, independent multiple factors, such as the previously mentioned:
Different manufacturers produce different precision ruler
Ruler different materials, different thermal expansion and contraction
When measured mood ups and downs
----------------
Disclaimer: This article is the original article CSDN bloggers "MA students of higher mathematics", and follow CC 4.0 BY-SA copyright agreement, reproduced, please attach the original source link and this statement.
Original link: https: //blog.csdn.net/ccnt_2012/article/details/81127117