Derivation notes of normal distribution

This article comes from Zhihu’s previous article on the derivation of normal distribution . I was enlightened, so I took notes.

from Introduction To The Normal Distribution (Bell Curve), BySaul Mcleod, PhD, https://www.simplypsychology.org/normal-distribution.html

Assume that there is an error probability density function $f\left(t\right)$ , now we have$Values\; of\; n$ independent observations$x_1$，$x_2$，$\cdots$，$x_n$, assuming the true value is $\mu$ , then the error is:

$$\begin{array}{rl}{e}_1& =x_1-m\\ e_2& =x_2-m\\ & ⋮\\ e_n& =x_n-\end{array}$$

According to life experience, this error $\epsilon$ , with a large number of observations, most of its values ​​should be$The\; range\; fluctuates\; around\; 0$ , and appears more frequently. For observations with large errors, the corresponding$|\epsilon |$ should also be large, and the frequency of occurrence should also be small. Make the maximum likelihood function:

$$\begin{array}{rl}L\left(\mu \right)& =\prod _{i=1}^{n}f\left(e_i\right)\\ & =f\left(x_1-m\right)f\left(x_2-m\right)\cdots f(x_n-m\end{array}$$

对$L\left(\mu \right)$ takes the natural logarithm:

$$\begin{array}{rl}\ln \left[L\right(\mu )]& =\ln \left[\prod _{i=1}^{n}f\left(e_i\right)\right]\\ & =\ln \left[f\left(x_1-m\right)f\left(x_2-m\right)\cdots f\left(x_n-m\right)\right]\\ & =\ln \left[f\left(x_1-m\right)\right]+\ln \left[f\left(x_2-m\right)\right]+\cdots +\ln \left[f\left(x_n-m\right)\right]\\ & =\sum _{i=1}^n\ln [f\left(x_i-m\right)\end{array}$$

In order to get $The\; maximum\; value\; of\; ln\left[L\right(\mu )]$ , for whichln ⁡ [ L ( μ ) ] \ln [L(\mu)]Find the partial derivative of$\ln$$[$$L$$($$\mu$$)]$$0$

$$\begin{array}{rl}\frac{\partial \ln \left[L\right(\mu )}{\partial \mu }& =\frac{\partial {\sum }_{i=1}^n\ln [f\left(x_i-m\right)}{\partial \mu }\\ & =-\sum _{i=1}^n\frac{f^{\prime }(x_i-m}{f\left(x_i-m\right)}\\ & =0\end{array}$$

令 $g(t)=\frac{f^{\prime }(t)}{f(t)}$, then the above formula becomes:

$$\sum _{i=1}^{n}g\left(x_i-m\right)=0$$

After reaching this step, the exciting part begins. This is also the brilliance of Gauss. He believes that μ \muThe unbiased estimate of$\mu$$\bar{x}$ , then the original formula becomes

$$\sum _{i=1}^{n}g\left(x_i-\bar{x}\right)=0$$

in,

$$\bar{x}=\frac{1}{n}\sum _{i=1}^n{x}_i$$

Solve the above equation for each $x_i$Find the partial derivative, for example, $x_1$Finding the partial derivative, we can get the following equation:

$$\begin{array}{rl}\frac{\partial {\sum }_{i=1}^{n}g\left(x_i-\bar{x}\right)}{\partial x_1}& =\frac{\partial {\sum }_{i=1}^{n}g\left(x_i-\frac{1}{n}{\sum }_{i=1}^n{x}_i\right)}{\partial x_1}\\ & =g^{\prime }\left(x_1-\bar{x}\right)\left(1-\frac{1}{n}\right)+g^{\prime }\left(x_2-\bar{x}\right)\left(-\frac{1}{n}\right)+\cdots +g^{\prime }\left(x_n-\bar{x}\right)\left(-\frac{1}{n}\right)\\ & =0\end{array}$$

将 $g^{\prime }\left(x_i-\bar{x}\right)$is regarded as an unknown number, and the above homogeneous linear equations are written as a matrix equation$\mathbf{Ax}=0$ form:

$$\left(\begin{array}{cccc}1-\frac{1}{n}& -\frac{1}{n}& \cdots & -\frac{1}{n}\\ -\frac{1}{n}& 1-\frac{1}{n}& \cdots & -\frac{1}{n}\\ ⋮& ⋮& ⋮& ⋮\\ -\frac{1}{n}& -\frac{1}{n}& -\frac{1}{n}& 1-\frac{1}{n}\end{array}\right)\left(\begin{array}{c}{g}^{\prime }\left(x_1-\bar{x}\right)\\ g^{\prime }\left(x_2-\bar{x}\right)\\ ⋮\\ g^{\prime }\left(x_n-\bar{x}\right)\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ ⋮\\ 0\end{array}\right)$$

For the coefficient matrix M \mathbf{M} of the above system of equations$M$ , put the 1st$1,2,3\cdots ,n$ lines are added sequentially to No.$1$ row, the following matrix can be obtained:

$$\mathbf{M}=\left(\begin{array}{cccc}1-\frac{1}{n}& -\frac{1}{n}& \cdots & -\frac{1}{n}\\ -\frac{1}{n}& 1-\frac{1}{n}& \cdots & -\frac{1}{n}\\ ⋮& ⋮& ⋮& ⋮\\ -\frac{1}{n}& -\frac{1}{n}& -\frac{1}{n}& 1-\frac{1}{n}\end{array}\right)\to \left(\begin{array}{cccc}0& 0& \cdots & 0\\ -\frac{1}{n}& 1-\frac{1}{n}& \cdots & -\frac{1}{n}\\ ⋮& ⋮& ⋮& ⋮\\ -\frac{1}{n}& -\frac{1}{n}& -\frac{1}{n}& 1-\frac{1}{n}\end{array}\right)$$

The first line is all 0, then $\mathrm{the}M=0$ , this only shows that the system of equations has infinite solutions, and specifically we need to calculate$\mathrm{rank}\left(M\right)$ . Ultimately, the solution to the above system of equations can be written as

$$\mathbf{X}=k\left(\begin{array}{c}{g}^{\prime }\left(x_1-\bar{x}\right)\\ g^{\prime }\left(x_2-\bar{x}\right)\\ ⋮\\ g^{\prime }\left(x_n-\bar{x}\right)\end{array}\right)=k\left(\begin{array}{c}1\\ 1\\ ⋮\\ 1\end{array}\right)$$

即 $g^{\prime }\left(x_1-\bar{x}\right)=g^{\prime }\left(x_2-\bar{x}\right)=\cdots =g^{\prime }\left(x_n-\bar{x}\right)=k$ , solving the differential equation, we can get:

$$g(t)=kt+b$$

Solve this differential equation:

$$\begin{array}{rl}\int \frac{f^{\prime }(t)}{f(t)}\mathrm{d}t=\int ktdt& \iff \int \frac{\mathrm{d}[f(t)]}{f\left(t\right)}=\frac{1}{2}k{t}^2+c\\ & \iff \ln [f(t)]=\frac{1}{2}k{t}^2+c\\ & \iff f\left(t\right)=K{e}^{\frac{1}{2}k{t}^2}\end{array}$$

At the same time, $f\left(t\right)$ is the probability density function, then it starts from$-\infty$ to$The\; integral\; of\; \infty$ is$1$ (regularity of probability density)

$$\begin{array}{rl}{\int }_{-\infty }^{+\infty }f(t)\mathrm{d}t& ={\int }_{-\infty }^{+\infty }K{e}^{\frac{1}{2}k{t}^{2dt}}\_\_\\ & =K{\int }_{-\infty }^{+\infty }{\mathrm{e}}^{-\frac{t^2}{2p^2}}\mathrm{d}t\\ & =K\sqrt{\sqrt{2}p\left[{\int }_{-\infty }^{+\infty }{e}^{-{\left(\frac{t}{\sqrt{2}p}\right)}^{2d}}\_\left(\frac{1}{\sqrt{2}p}t\right)\right]\left[\sqrt{2}p{\int }_{-\infty }^{+\infty }{e}^{-{\left(\frac{s}{\sqrt{2}p}\right)}^{2d}}\_\left(\frac{1}{\sqrt{2}p}s\right)\right]}\\ & =K\sqrt{2}p\sqrt{{\int }_{-\infty }^{+\infty }{\int }_{-\infty }^{+\infty }{e}^{-\left(u^2+v^2\right)}\mathrm{d}u\mathrm{d}v}\\ & =K\sqrt{2}p\sqrt{{\int }_0^{2}di{\int }_0^{+\infty }{e}^{-r^2}rdr}\\ & =K\sqrt{2}p\sqrt{Pi}\\ & =1\end{array}$$

Finally, the probability density function is obtained:

$$f(t)=\frac{1}{\sqrt{2p.m}p}{\mathrm{e}}^{-\frac{1}{2}{\left(\frac{t}{p}\right)}^2}$$