Maximum likelihood estimation of signal processing (self-study)
maximum likelihood estimation
Basic idea:
Without any prior knowledge of the unknown quantity (or parameter) being estimated, use several known observations to estimate the parameter.
Therefore, when using the maximum likelihood estimation method, the estimated parameters are assumed to be constant and unknown, and the known observation data are random variables.
Suppose the N observed values of the random variable a i=2> Therefore, likelihood function is and maximum likelihood estimation is to find Therefore, the maximum likelihood estimate can also be regarded as the global maximum point of the likelihood function.
log likelihood function
is recorded as
Therefore, the optimization condition for the maximum likelihood estimation of θ is
Under general conditions, θ is a vector parameter, and the formula is < /span> and finally solved by the formula
If x1,...,xN are independent observation samples, the likelihood function is written as
Maximum likelihood estimation properties:
(1) Maximum likelihood estimation is generally not unbiased, but its bias can be corrected by multiplying the estimated value by an appropriate constant;
(2) The maximum likelihood estimate is a consistent estimate;
(3) The maximum likelihood estimate gives an estimate of superiority, if it exists;
(4) For large N, the maximum likelihood estimate is Gaussian distribution, and its mean is θ and the variance is
References:
[1] "Modern Signal Processing (Third Edition)" Zhang Xianda