estimate
Life, we often estimate the number of values, such as how long to go from home to school? Chinese cabbage about how many pounds? Why estimate the specific value it? "Estimate" is not a shot in the dark, it is based on existing data. Round trip from home to school many times, the hands also took numerous pieces of cabbage, then we will calculate an approximate value with scale in mind.
Moment Estimation
Moments estimates that moment estimation method, also known as the "moment method estimates", is a method to estimate the expected value of the use of the existing sample.
A mathematical problem mathematical characteristics desired objective existence, is a specific value, but this value is calculated it needs to know some of the "known conditions" and that these conditions are known in the real world is not known. Fortunately, we can always get some random sample, use these samples to estimate a value:
Hat equals sign indicates that estimate. Each of the X- i is a simple random sample, and we believe that each sample are equally likely, this is actually the real world of one of last resort approach. Under the action under the law of large numbers, this estimate will gradually stabilize, close to the real value.
There are A, B and two shooting athletes standing in front of us, their grade point average is not attached to the body, how to judge their results?
A line with empirical approach is to let them play 10 each gun, and then calculate the mean. For example, the X- i is i Hill Classic gun results, we estimate that for A (the X- 1 + the X- 2 + ... + the X- 10 ) / 10. Here is a simple use of the mean, and there is no probability involved, because we do not know the probability of each ring of armor play, had considered equally weighted average. Mathematical expectation is true athlete performance, we need to calculate the mathematical expectation of the probability of each ring known athletes play, but "known" does not always exist in the real world, so just settle for using "estimates . "
Independent and identically distributed
Iid is a concept in probability theory, that is, a set of data among each other without disturbing each other, randomly appear in the real environment.
Independent has been mentioned numerous times in the shooting competitions every shot is independent, will not (change the athlete's mental state aside) because under this influence the result of a shot. If it is to take a black ball from a pile of white ball, white ball with the reduction of the probability of the next ball will be taken black gets bigger, you can not say to take the ball every time behavior independently of each other.
"The same profile" means each time taken from a particular set of results, such as craps, each time taken from 1 to 6 the results, the same samples are called distributed. If mixed with several dice surface 12, the sample is not identically distributed.
Unknown density function
In the continuous variables, as long as we know the probability density f (x) variable, you can know its expectations:
The problem is that f (x) is usually unknown, only know its model, but are unsure of the specific model parameters. We set up this unknown parameter θ, the probability density f (x; θ), f represents the affected θ, the mathematical expectation formula:
Θ is actually a vector, for example:
Exemplary probability density provided that continuous random variable 
, the estimated amount of torque required θ.
X may first calculate the Moment Estimation:
Only 0 <x <1 is to be calculated when θ:
Maximum Likelihood
Maximum likelihood estimation method (Maximum Likelihood Estimate, MLE), also known as maximum likelihood estimation or maximum likelihood estimation, is to establish a statistical method of maximum likelihood principle on the basis of.
Maximum Likelihood meaning
"Likelihood" is the "possibility" means. We often hear "maximum likelihood", a term derived from the actual, the following figure explains its meaning.
A, B are two identical boxes, A has 100 white balls and a black ball, B has 100 black balls and a white ball. Now free to be removed from two boxes in a small ball, the result is a black ball, the black ball is removed from the box in which? The first reaction is the "most likely taken from B," which is generally in line with experience. Here the "most likely" is the "maximum likelihood" means.
The likelihood and the likelihood function
假设有一个独立同分布的数据集X，它的参数是θ。现在从X中取出一些样本x={ x1, x2, …, xn}，P(x;θ)表示给定参数θ时，从X中取得这些样本x的可能性：
其中P(x;θ)类似于条件概率，但不等于条件概率，因为θ只是一个密度函数中的参数，并不是一个事件。
假设现在θ有两个取值θ1和θ2，对于X中的一些样本x={ x1, x2, …, xn}，如果P(x, θ1 )> P(x, θ2 )，就认为θ1对产生x的可能性（似然性）要大于θ2，P(x, θ1 )和P(x, θ2)就是似然，是对参数θ产生样本x的可能性的度量。
还是以射击为例，假设按运动员的成绩由高到低分为一级、二级、三级，甲打出了10枪x={9,9,10,10,8,9,9.5,9.5,9.5,9}。运动员的级别相当于影响成绩的参数θ，当θ等于一级时，甲打出这个成绩的可能性较高。
现在需要根据给定样本x来求P(x; θ)，由于样本是已知的，将所有x的值代入上面的公式，将得到一个只有θ的式子，这个式子称为θ的似然函数，记为L(x;θ)或L(θ)：
最大似然估计
知道了似然函数，最大似然估计就很容易理解了：对于一个给定的样本集，挑选使得P(x;θ)能够达到最大时的参数 作为θ的估计值，使得：
最终将求得θ的一个估值 ，在 时，似然函数的值最大。
极值点通常是在导数等于0的点取得，因此可以通过下式求得θ：
如果θ是n维向量，则：
对于一些特殊的密度函数（比如指数密度函数）来说，直接求dL/dθ太过繁琐，由于L与lnL在同一θ处取到极值，所以也经常使用：
示例
示例1
设样本的总体分布率为：P{X=x}=px(1-p)1-x，求p在观察样本{ x1, x2, …, xn }下的最大似然估计量。
这里只不过是把θ用p表示，现在我们做一下替换，变成熟悉的形式：
L(θ)是θ的指数形式，换成对数更为简单：
根据对数的基本公式继续计算：
示例2
总体样本服从参数为λ的指数分布，{x1, x2, …, xn}是观察样本，求λ的最大似然估计值。
总体样本的概率密度是：
作者：我是8位的
出处：http://www.cnblogs.com/bigmonkey
本文以学习、研究和分享为主，如需转载，请联系本人，标明作者和出处，非商业用途！
扫描二维码关注公作者众号“我是8位的”
