Depth learning / machine learning basic arithmetic knowledge finishing (VIII): central limit theorem, univariate and multivariate Gaussian distribution

Central Limit Theorem

Random variable X1, X2, ... Xn, independent and identically distributed, and have limited mathematical expectation and variance: $E (X_i) = \ mu$ , $D(X_i)=\sigma^2$ , then for any real number x, the distribution function

satisfies

this theorem, when a large n, a random variable

approximately follow a standard normal distribution N (0,1). Thus, when a large n,

approximately normal distribution $N (nμ, nσ2)$ . The theorem of the central limit theorem is the simplest and most common form, in practical work, as long as n is large enough, they can be independent and identically distributed random variables and as a normal variable. This method is very common in mathematical statistics used when dealing with large sample, it is an important tool.

Simple application of the Central Limit Theorem

References [1]

Gaussian distribution

Gaussian distribution Gaussian distribution, also known as being too distributions Normal distribution, is a very important in mathematics, physics and engineering areas such as probability distributions, has a major influence on many aspects of statistics.

One yuan Gaussian distribution

If the random variable conforms to a Gaussian distribution $X \ sim N (\ mu, \ sigma ^ 2)$，则有如下的概率密度函数

满足

而如果我们对随机变量 $X$进行标准化 $Z = \frac{X-\mu}{\sigma }$, 那么变量 $Z$服从0均值，1方程的一元标准高斯分布。

多元高斯分布

多维高斯分布的公式：

其中 $x=(x_1,x_2,...,x_n)$为一个n维向量， $\mu$是均值向量， $\sum$是协方差矩阵。

多元高斯分布的的线性变换

两个高斯分布的KL散度

参考资料[5]

两个一元（一维）高斯分布的KL散度 $KL(p_1||p_2)$ :

KL divergence two multi-dimensional Gaussian distribution $KL(p_1||p_2)$ :

This algorithm will be used in VAE, record it, if we look at VAE time can be found.

Reference material

[1] the central limit theorem, Baidu Encyclopedia
[2] https://zhuanlan.zhihu.com/p/38501770
[. 3] https://zhuanlan.zhihu.com/p/58987388
[. 4] HTTPS: // zhuanlan. zhihu.com/p/90272131
[. 5] of VAE (. 1) - Starting from KL