Some basic knowledge about probability theory

probability space

Random experiment is the basic concept of probability theory. The result of experiment cannot be expressed in exact language, but it has the following three characteristics: ( 1)
It can be repeated under the same conditions;
Know all possible results of the experiment;
(3) Before each trial, it is not possible to determine which result will appear.

The set of all possible results of a random experiment is called the sample space or basic event space of this experiment , denoted as $Oh$。Oh \OmegaThe element e in$\Omega$ is called sample pointorbasic event,Ω \OmegaA subset event A of$\Omega$ is called an event, and the sample space$\Omega$ is calledan inevitable event, and the empty set$\varnothing$ is calledan impossible event.

Definition 1 : Let $\Omega$ is a set, F isΩ \OmegaA family of sets consisting of some subsets of$\Omega \; .$If
(1)$Oh\in F$
（2）若$A\in F$ ,则$\tilde{A}=Oh\backslash A\in F$
（3）若$A_n\in F,n=1,2,...$，则${\cup }_{n=1}^{\infty }{A}_n\in F$
is called$p^{-}$ Algebra (Borel fields). $(\Omega ,F)$ is a measurable space, and the elements in F are called events.
It is easy to know from the definition:
(4)$\varnothing \in F$
（5）若A,B$\in$F，则$A\backslash B\in F$
（6）若$A_i\in F,n=1,2,.\_\_$ _${\cup }_{i=1}^{\infty }{A}_i$，${\cap }_{i=1}^{\infty }{A}_i$，${\cap }_{i=1}^{\infty }{A}_i\in F$

Random Variables and Their Distributions

Random variables are the main research objects of probability theory, and the statistical laws of random variables are described by distribution functions.

Definition 4 : Let $(Oh,F,P)$ is the probability space,$X=X\left(e\right)$ is defined in$A\; real\; function\; on\; \Omega$ , if for any real number$x,\{and：X(and)\le x\}$ , then$X\left(e\right)$ is a random variableon F, abbreviated as random variable$X$，称
$F(x)=P(e:X(and)\le x),\infty <x<\infty$ is the distribution function
of the random variable.

The distribution function has the following properties:
(1) $F\left(x\right)$ is a non-decreasing function, that is, when$x_1<x_2$, there is $F(x_1)\le F(x_2)$.
（2）$F(-\infty )={\lim }_{x\to -\infty }F(x)=0,F(\infty )={\lim }_{x\to \infty }F\left(x\right)=1$.
（3）$F\left(x\right)$ is right continuous, that is,$F(x+0)=F(x)$.

The probability distribution of a discrete shorthand variable X is described by a distribution column :
$p_k=P(X=x_k),k=1,2,...Its$
distribution function
$F\left(x\right)={\sum }_{x_k\le x}{p}_k$
Probability distribution of continuous random variable X uses probability density $f\left(x\right)$ description, its distribution function
$F(x)={\int }_{-\infty }^{x}f(t)dt$

Common random variable distribution tables are as follows:

Definition 5 : Let $(\Omega ,F,P)$ is a probability space,$X=X\left(and\right)=(X_1(e),...X_n(e\left)\right)$ is defined in$On\; \Omega$ in n-dimensional space$R^{}$A vector function of values ​​in${}^n$ , if for any x = ( x 1 , x 2 , . . . xn ) ∈ R n , { e : X 1 ( e ) ≤ x 1 , X 2 ≤ x 2 , . . . , X n ( e ) ≤ xn } ∈ F x=(x_1,x_2,...x_n) \in R^{n}, \{ e:X_1(e)\leq x_1,X_2 \leq x_2,.. .,X_n(e)\leq x_n\} \in Fx=(x1​,x2​,...xn​)∈Rn,{ e:X1​(e)≤x1​,X2​≤x2​,...,Xn​(e)≤xn​}∈F , then$X=X\left(e\right)$ is ann-dimensional random variableorn-dimensional random vector, which is called
$F(x)=F(x_1,x_2,...x_n)=P(e:X_1(e)\le x_1,X_2\le x_2,...,X_n(e)\le x_n),x-(x_1,x_2,...x_n)\in R^n$
为$X=(X_1,X_2,...X_n)$ jointdistribution function.

Numerical properties of random variables

The probability distribution of a random variable is completely described by its distribution function, but how to determine the distribution function is quite troublesome. In practical problems, sometimes we only need to know some eigenvalues ​​of random variables.
Definition 7 : Let the distribution function of random variable X be $F(x)$，若${\int }_{-\infty }^{\infty }|x|dF(x)<\infty$ , then
$EX={\int }_{-\infty }^{\infty }xdF\left(x\right)$ is the numerical expectationormeanof
X. The integral on the right-hand side of the above equation is called the Lebesgue-Stieltjes integral. If X is a discrete random variable, the distribution ispk = P ( X = xk ) , k = 1 , 2 , . . . p_k=P(X=x_k),k=1,2,...

$p_k=P(X=x_k),k=1,2,...$
Then
$EX={\sum }_{k=1}^{\infty }{x}_k{p}_k$
If X is a continuous random variable, the probability density is $f(x)$，则
$EX={\int }_{-\infty }^{\infty }xf\left(x\right)dx$
The mathematical expectation of a random variable is the average of the values ​​of the random variable according to the probability.

Definition 8 : Let X be a random variable, if $E{X}^2<\infty$ , abbreviated$DX=E\left[\right(X-EX{)}^2]$is the variance of X.
The mathematical expectation of a random variable is the degree to which the value of the random variable deviates from the mean.

Definition 9 : Suppose X, Y are random variables, $E{X}^2<\infty$,$E{Y}^2<\infty$ ,base
$B_{XY}=E\left[\right(X-EX)(Y-EY\left)\right]$
is the covariance of X and Y, which means
$r_{XY}=\frac{B_{XY}}{\sqrt{DX}\sqrt{DY}}$
若$r_{XY}$If it is 0, it means that X and Y are not correlated, and the correlation coefficient indicates the degree of linear correlation between X and Y.

Several properties that need attention:
(1) $E(toX+bY)=aEX+bEY$ , where a and b are constants;
(2) If X and Y are independent, then$And\left[XY\right]\_=EXEY$ ;
(3) If X and Y are independent, then$D(aX+bY)=a^{2}DX+b^{2}DY$, where a, b are constants

conditional expectation

Let X and Y be discrete random variables. For a given y, if $P\{AND=y\}>0$则称
P { X = x ∣ Y = y } = P { X = x , Y = y } P { Y = y } P\{X=x|Y=y\}=\frac{P\{X =x,Y=y\}}{P\{Y=y\}}P{ X=x∣Y=y}=P { Y = Y }P{ X=x,Y=y}​
For a given $Y=$conditional probabilityof X at$y$

给定$Y=y$ ,the conditional distribution functionis:
F ( x ∣ y ) = P { x ≤ x ∣ Y = y } , x ∈ RF(x|y)=P\{x \leq x| Y = y\} , x \in RF(x∣y)=P{ x≤x∣Y=y},x∈R

And given $Y=y$ ,the conditional expectationis:
E [ X ∣ Y = y ] = ∫ xd F ( x ∣ y ) = ∑ xx P { X = x ∣ Y = y } E[X|Y=y]= \int xdF(x|y)=\sum_{x}xP\{ X=x|Y=y\}E[X∣Y=y]=∫xdF(x∣y)=∑x​xP{ X=x∣Y=y}

If X and Y are continuous random variables, their joint probability density is $f(x,y)$ , then f Y ( y ) > 0 f_Y(y)>0for everything$f_Y(y)>0by$ y,by$Y=y$ ,the conditional probability densityis defined as:
$f(x|y)=\frac{f(x,y)}{f_Y\left(y\right)}$

给定$Y=y$ ,the conditional distribution functionof
X is F ( x ∣ y ) = P { X ≤ x ∣ Y = y } = ∫ − ∞ xf ( u ∣ y ) du F(x|y)=P\{X\leq x| Y=y\}=\int^{x}_{-\infty}f(u|y)duF(x∣y)=P{ X≤x∣Y=y}=∫−∞x​f ( u ∣ y ) d u

给定$Y=y$ ,the conditional expectationis defined as
$E[X|Y=y]=\int xdF(x|y)=\int xf(x|y)$

From this it can be seen that the definition is now exactly as in the unconditional case, except that probability is the conditional probability of the event {Y=y}.

E[X|Y=y] is a function of y, where y is a possible value of Y. If the mean value of X is fully considered under the condition of known Y, it is necessary to replace y with Y, then E[X|Y=y] is a function of random variable Y, which is also a random variable, called X under Y conditional expectations .

Properties :
If random variables X and Y are expected to exist, then
$EX=E\left[E\right(X|Y)]=\int E[X|Y=y]d{F}_Y\left(y\right)$
If Y is a discrete random variable, the formula can be expressed as
$EX={\sum }_{y}E[X|Y=and\left]P\right\{AND=y\}$
If Y is a continuous random variable with probability density$f(y)$
$EX={\int }_{-\infty }^{\infty }E[X|Y=y\left]f\right(y)dy$