常见概率分布总结

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/crazy_scott/article/details/82818042

Discrete

Bernoulli distribution

• pmf
  • $f_X(x) = P(X= x) =\left\{\begin{aligned}(1-p)^{1-x}p^x & \quad \text{for x = 0 or 1}\\ 0 & \quad\text{otherwise}\end{aligned}\right.$
• expectation
  • $E(X) = p$

Binomial distribution

• pmf
  • $f_X(k) = P(X= k) =\left\{\begin{aligned}C_n^kp^k(1-p)^{n-k} & \quad \text{for k=0,1,....,n}\\ 0 & \quad\text{otherwise}\end{aligned}\right.$
• expectation
  • $E(X) = np$
• variance
  • $var(X) = np(1-p)$

Geometric distribution

• pmf
  • $f_X(k) = P(X= k) =\left\{\begin{aligned}p(1-p)^{k-1} & \quad \text{for k=1,2,3...}\\ 0 & \quad\text{otherwise}\end{aligned}\right.$
• expectation
  • $E(X) = \frac{1}{P}$

Negative binomial distribution

• The negative binomial distribution arises as a generalization of the geometric distribution.

• Suppose that a sequence of independent trials each with probability of success $p$ is performed until there are $r$ successes in all.

  • so can be denote as $p \cdot C_{k-1}^{r-1} p^{r-1}(1-p)^{(k-1)-(r-1)}$

• pmf

  • $f_X(k) = P(X= k) =\left\{\begin{aligned}C_{k-1}^{r-1}p^r(1-p)^{k-r} & \quad \text{for k=1,2,3...}\\ 0 & \quad\text{otherwise}\end{aligned}\right.$

Hypergeometric distribution

• Suppose that an urn contains $n$ balls, of which $r$ are black and $n-r$ are white. Let $X$ denote the number of black balls drawn when taking $m$ balls without replacement.
• pmf
  • $f_X(k) = P(X= k) =\left\{\begin{aligned}\frac{C_r^kC_{n-r}^{m-k}}{C_n^m} & \quad 0\le k \le r\\ 0 & \quad\text{otherwise}\end{aligned}\right.$

Possion distribution

• can be derived as the limit of a binomial distribution as the number of trials approaches infinity and the probability of success on each trial approaches zero in such a way that $np = \lambda$, $\lambda$ can be seen as the successful trials
• pmf
  • $P(X = k) = \frac{\lambda^k }{k!} e^{-\lambda} \quad k = 0,1,2...$

Continuous

Uniform distribution

• A uniform r.v on the interval [a,b] is a model for what we mean when we say “choose a number at random between a and b”
• pdf
  • $f_X(x) = \left\{\begin{aligned}\frac{1}{b-a} & \quad a\le x \le b\\ 0 & \quad\text{otherwise}\end{aligned}\right.$

Exponential distribution

• Exponential distribution is often used to model lifetimes or waiting times, in which context it is conventional to replace $x$ by $t$.
• pdf
  • $f_X(x) = \left\{\begin{aligned}\lambda e^{-\lambda x} & \quad x\ge 0\\ 0 & \quad\text{otherwise}\end{aligned}\right.$
• cdf(easy to get)
  • $F_X(x) = \left\{\begin{aligned}1-e^{-\lambda x} & \quad x\ge 0\\ 0 & \quad\text{otherwise}\end{aligned}\right.$
• expectation
  • $E(X) = \lambda$
• variance
  • $var(X) = \lambda^2$

property

• let $X,Y$ are independent Poisson r.v.s with $\theta_1,\theta_2$,then $X+Y\sim Poisson (\theta_1+\theta_2)$

Gamma distribution

• pdf
  • $g(t) = \left\{\begin{aligned}\frac{\lambda^\alpha}{\tau (\alpha)}t^{\alpha-1}e^{-\lambda t} & \quad t\ge 0\\ 0 & \quad\text{otherwise}\end{aligned}\right.$
• $\tau(x) = \int _0^\infty u^{x-1}e^{-u}du,x>0$
• expectation
  • $E(X) = \frac{\alpha}{\lambda}$
• variance
  • $Var(X)= \frac{\alpha}{\lambda ^2}$

Property

• Note that if $\alpha = 1$, the gamma density coincides with the exponential density.
• conduct
  • $\because \tau(\alpha ) =\int _0^\infty x^{\alpha-1}e^{-t}dx$
  • $\therefore x = \lambda t,\to \tau (\alpha) = \lambda^\alpha \int _0^\infty t^{\alpha-1}e^{-\lambda t}dt$
  • $\therefore \frac{1}{\tau (\alpha)}\lambda^\alpha \int _0^\infty t^{\alpha-1}e^{-\lambda t}dt = 1$
  • $\therefore g(t) =\frac{\lambda^\alpha}{\tau(\alpha)}t^{\alpha-1}e^{-\lambda t}$
• $\alpha$ is called a shape parameter for the gamma density,
• Varying $\alpha$ changes the shape of the density
• $\lambda$ is called a scale parameter
• Varying $\lambda$ corresponds to changing the units of measurement and does not affect the shape of the density
• how to understand gamma?

Normal distribution

• pdf
  • $g(t) = \left\{\begin{aligned}\frac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/(2\sigma^2)} & \quad t\ge 0\\ 0 & \quad\text{otherwise}\end{aligned}\right.$
• $\mu$ is the mean
• $\sigma$ is the standard deviation
• If $X \sim N(\mu; \sigma^2)$ ,and $Y = aX + b$, then $Y \sim N(a\mu+b,a^2\sigma^2)$
  • especially, if $X \sim N(\mu,\sigma^2)$, then $Z = \frac{x-\mu}{\sigma}\sim N(0,1)$
• $aX+bY \sim N(a\mu_X+b\mu_Y,a^2\sigma_X^2 + b^2\sigma_Y^2 + 2ab\rho \sigma_X\sigma_Y)$

property

• if $X,Y \sim N(0,1)$,then $U = \frac{X}{Y} $ is Cauchy r.v (lec3)
  • $f_U(u) = \frac{1}{\pi (u^2+1)}$

Exponential family

• A family of pdfs or pmfs is called an exponential family if it can
  be expressed as:
  • $p(x,\theta) = H(x)\exp(\theta^T \phi(x) - A(\theta))$
  • $H(x)$ is the normalization factor
• It is very helpful to model heterogeneous data in the era of big data.
• Bernoulli, Gaussian, Binomial, Poisson, Exponential, Weibull, Laplace, Gamma, Beta, Multinomial, Wishart distributions are all exponential families
• the explain can be seen here

Property

• $E(X) = E(E(X|Y))$
  • 可以理解为先分组求期望，与直接求期望一样
• $Var(X) = E(Var(X|Y)) + Var(E(X|Y))$
  • 可以理解为组内方差的期望 + 组间方差