Reference: https://www.zhihu.com/question/20447622
The maximum likelihood estimation and the least square method feel similar in many places, but they feel different. The learning process is a bit confusing. It feels good to find a Zhihu answer here, but it is forbidden to reprint it, hahahaha, Everyone should look at the reference website. There is another person below. You can understand this after reading the answer on that website. To sum it up:
Maximum likelihood estimation: Now that you have got a lot of samples (all dependent variables in your data set), these sample values have been achieved, and maximum likelihood estimation is to find the parameter estimate (group), so that the sample value that has been achieved before The probability of occurrence is greatest. Because the sample you have on hand has already been realized, it is logical to have the greatest probability of occurrence. At this time, the joint probability of all observations in the sample is maximized, which is a continuous product. As long as the logarithm is taken, it becomes a linear addition. At this time, by taking the derivative of the parameter and setting the first derivative to zero, the maximum likelihood estimate can be obtained by solving the equation (set).
Least squares: Find a (group) estimated value that minimizes the distance between the actual value and the estimated value. It is the most ideal to use the absolute value of the difference between the two to summarize and minimize it. However, it is more troublesome to find the minimum value of the absolute value mathematically. The sum of the squares of the difference is the smallest value, which is called the least squares. "Two multiplication" in English is least square, in fact, the literal meaning in English is "the smallest square". At this time, take the derivative of the sum of the squares of the difference with respect to the parameter, and take the first derivative as zero, which is OLSE.