Anatomy of the Kalman family from scratch-(01) Preliminary knowledge points

Explaining the summary link of a series of articles about slam:The most comprehensive slam in history starts from scratch, for the Kalman family explained in this column Zero anatomy link:The Kalman family’s anatomy from scratch - (00) catalog latest explanation without blind spots: https://blog.csdn.net/weixin_43013761/article/details/133846882

${\color{blue}{The center directly below the end} provides my \color{red} contact information, \color {blue}Click on your photo to display WX→official certification}$

$\color{red} Solemnly declare: This series of blogs are exclusively owned by me (WenhaiZhu). Reprinting and plagiarism are prohibited. , thank you for the first report!$

I. Introduction

In the subsequent process, various fragmentary knowledge points will inevitably be involved, such as the recording or derivation of some formulas in probability theory. If a large amount of space is spent on explaining each time, there will be too many blogs. , and the knowledge points will be relatively messy, so this blog is mainly used to record the conclusions of some important formulas. Each formula will have a corresponding number, but it should be noted that $order\ color{red} numbers may not be in order$ . Some may have a derivation process, some may only have reference links, and of course some may only have formulas, which means it should be relatively simple. You can easily find the corresponding derivation process on Baidu or Google.

1. Probability related

$\color{blue}(1) Random variable:$ Large copy $X$ , $Y$ ， $Z$ … represents a random variable (event); lowercase $x$ , $y$ ， $z$ represents the specific value of the random variable, usually a specific numerical value; such as $X$ represents a random event whether it will rain tomorrow, $X = 1$ Display below rain, $X = 0$ It's raining all the time.

$\color{blue}(2) Discrete probability:$ for jogao $P$ represents discrete probability probability, here used $Y$ represents a random event of rolling the dice, then $P (Y = 1)$ represents the probability that the die is 1, such as $P(Y=1)=\frac{1}{6}$ , then the probability that the die is 1 is $\frac{1}{6}$ , of course it can also be expressed using mathematical formulas. Such as $P(Y=y)=e^{-\lambda} \frac{\lambda^{k}}{k ! }$ 。

$\color{blue}(3) Continuous probability:$ The discrete probability can be understood as a histogram distribution, then the continuous probability is the ratio of the area enclosed by an interval to the entire area, such as $P(1<Y<5)=\frac{3}{6}$ can represent the probability of rolling a die and getting 2, 3 or 4. Since it is an area, you can use integrals. Another example: $P(2<+∞)=\int_{2}^{- \infty} \frac{1}{\sqrt{2 \pi}} e^{-\frac{x^{2}}{2}} \mathrm dx$

$\color{blue}(4)Joint probability:$ Two random events can be expressed jointly, such as $P (X = x, AND = y)$ , display incident $X$ appearance $x$ Same time incident $Y$ 出现 $The probability of y$ . For example, if two dice are thrown, then $P (X = 1, AND = 5)$ means the probability that the result of the first dice is 1 and the result of the second dice is 5.

$\color{blue}(5)Conditional probability:$ First of all, we understand it from the naming point of view, that is, under the premise (condition) of the occurrence of an event, the probability of another event occurring. Here is an example $P (Y = 1∣ X = 5)$ represents the probability that the result of the second die is 1 under the premise that the result of the first die is 5.

$\color{blue}(5)Probability density (PDF):$ Let’s talk about probability density function here. We write the [(3) continuous probability] expression mentioned above as a general expression as follows: $P (a \leq X \leq b) = \int_{a}^{b} f (x) d x$ Primary name $X$ is a continuous random variable, $f (x)$ 为 $The probability density function of X$ is referred to as probability density or density. $\Delta{x} < /span>$ 表示 $\Delta{x}$ The probability of this interval. Probability density can be viewed as probability about $x$ Target number, attention to demand $x$ 与 $f (x)$ The city is connected.

$\color{blue}(6) Edge probability:$ For one-dimensional marginal probability (Marginal Probability), it is the probability of a certain event occurring, regardless of other events, such as in the dice example $P (X = 1)$ 与 $P (X = 2)$ . Multi-dimensional edge probabilities will be more complicated, and there are differences between discrete and continuous edge probabilities, and they also involve joint probability and total probability formulas, etc., so they will be explained separately later. In addition, although the one-dimensional edge probability can be passed $\Delta{x}$ Indication, however, there is no actual meaning, usually unforeseen.

$\color{blue}(7) Prior and posterior:$ To be compatible with most books or bloggers, use $\check {a}$ or $a^-$ Display destination, $\hat {a}$ or $a^+$ After display, as shown $\check {x}$ given $x^-$ represents the prior state, $f^+_X(x)$ 与 $\hat f_X(x)$ represents the posterior probability density function. There is no difference between them. The way of writing is just different.

2.Matrix correlation

$\color{blue}(1)Variable:$ Afterwards, a small copy $a$ 、 $b$ 、 $c$ represents a scalar value, the lowercase letters after bolding $\mathbf a$ 、 $\mathbf b$ 、 $\mathbf c$ Equal display direction, then large photographic $\mathbf A$ and bold indicates the matrix.

$\color{blue}(2)Matrix:$ In addition, some special explanations are needed about the matrix, such as $\mathbf A_a^b$ , which means that it is a transformation matrix, and its meaning represents the coordinate transformation from system a to system b.

2. Conditional probability (multidimensional)

$\color{blue}(1) Conditional probability (discrete):$ If random event $X$ , $Y$ is a discrete event, then the following formula can be derived:
$\color{Green} \tag{01} P(X=x \mid Y=y)=\frac{P(X=x, Y=y) }{P(Y=y)}$ This is quite easy to understand, just change it $P (X = x ∣ AND = y) P (Y = y) = P (X = x, AND = y)$ . It can be understood by drawing, two events $X, Y$ are represented by ellipses respectively, then their intersection is ${P(X=x, Y=y)}{P(Y=y)}$ 。

$\color{blue}(2) Conditional probability (continuous):$ If it is continuous, it cannot be expressed in a similar discrete way. The one-dimensional continuous edge probability was mentioned earlier, as in the above formula $P (Y = y)$ is meaningless, so we should express it through probability density. Here we use $f_X(x)$ 、 $f_Y(y)$ Display connection amount $X, The probability density function of Y$ . Then $f_{YX}(y|x)$ Approximate display condition $P (Y ∣ X)$ density function. Arisuishi formula below:
$\color{Green} \tag{02} f_{YX}(y|x)=\frac{f(X=x, Y=y)}{f_X(x)}$ So what does this probability density do? Like the previous one-dimensional one, the conditional probability can be obtained by integration $P (Y ∣ X)$ . Backward progress analysis.

3. Total probability formula-marginal probability-multidimensional

$\color{blue}(1) Total probability (discrete):$ Suppose there are random events (variables) $X, Y$ ，Washido $X$ Approximate probability $P(X_i)$ ；Hereafter, conditional probability $P(Y|X_i)$ ，Owned by $X$ 情况 $X_0,X_1,....X_i]$ occurs, its $Y$ The probability that the event also occurs. Here we only consider two dimensions, that is, the events are only $X, Y$ , let’s put all $P(Y|X_i)$ $P (Y)$ Official formula: $\color{Green} \tag{03} P(Y) =\sum_iP(X_i)P(Y|X_i)$ The above formula is relatively easy to understand. For example, there are five in the box To find the color (red, yellow, blue, green, cyan), now take two balls in a row and record them as events $X, Y$ , use (red|yellow) to represent the first time you take out yellow and the second time you take out red, and so on, if you know $P (red ∣ red)$ , $P (red ∣ yellow)$ , $P (red ∣ 蓝)$ , $P (red ∣ green)$ , $P (red ∣ blue)$ Approximate probability, same time and distance $P(X_i)=\frac{1}{5 }$ , then according to the above formula we can solve the probability that the ball is red when we take it out once, and the marginal probability $P (Y) = Marginal probability of red$ . Of course, it can also be obtained similarly
$P (Y) = Yellow$ ， $P (Y) = The marginal probability of blue$ and so on. The principle is to combine all $X_i$ occurs simultaneously $Y_j$ The probabilities of are accumulated, and then the marginal probability $Y_j$ (The above formula is derived for simplicity, omitting $Y$ 的下标 $j$ )。

$\color{blue}(2) Full probability (continuous):$ The continuous situation is much more complicated, unlike discrete, which can be directly accumulated. It is relatively complicated, mainly because in the discrete case $P(Y_j)$ is meaningless. However, the density function mentioned earlier is meaningful and useful. For this use Newton-Leibniz's previous probability density $P(a≤X≤b)=\int_{a }^{b}f(x)\mathrm dx$ Reprinted, possible $F_X(-\ infty≤x)=\int_{-\infty}^{x}f_X(x)dt$ 。设密度函数 $f_{YX}(y,x)$ 、 $f_X(x)$ 、 $f_Y(y)$ ，也就是说 $F_X(x))^{'}=f_{X}(x)$ 、 $F_Y(y))^{'}=f_{Y}(y)$ . Then the following formula can be derived further:
$\color{Green} \tag{04} F_X(x)=F_X(-\infty≤x)=\int_{-\infty}^{x}[\int_{-\infty}^{\infty}f (x,y)\mathrm dy]\mathrm dx$ 首先 $f (x, y)$ 时关于 $x, Density function of y$ , aligned variables $y$ 的积分 $\int_{-\infty}^{\infty}f(x,y)\mathrm dy=f(x)$ , here from infinity to The integral of positive infinity, that is, $All possibilities for y$ are considered. About $Integration of y$ , integration result $y$ Of course not, as follows: $\color{Green} \tag{05} f_X(x)=\int_{-\infty}^{\infty}f(x,y)\mathrm dy$ It's just the same as below $x$ density function $f (x)$ , after doing By integrating twice, we get $F_X(x)=F_X(-\infty≤x)$ . Of course you can hold it $x, By exchanging the integrals of y$ , we can get: $\color{Green} \tag{06} F_Y(y)=F_y(-\infty≤y)=\int_{-\infty} ^{y}[\int_{-\infty}^{\infty}f(y,x)\mathrm dx]\mathrm dy$ $\color{red}注意:$ According to the calculated $F_X(x)=F_X(-\infty ≤x)$ 与 $F_Y( y)=F_y(-\infty≤y)$ We can calculate any edge probability, such as $F_{X_{ij}} (x \in [x_{i}, x_{j}]) = F_{X} (X_{i}) - F_{X} (X_{j})$ 。

4. Bayes’ formula

1. Discrete derivation

根据【条件概率(离散)】(01) 式与【全概率(离散)】(03)式子，可以推导出：
$\color{Green} \tag{07} P\left(X_{i} \mid Y\right)=\frac{P\left(Y \mid X_{i}\right) P\left(X_{i}\right)}{P(Y)}=\frac{P\left(Y \mid X_{i}\right) P\left(X_{i}\right)}{\sum_{i=1}^{n} P\left(Y \mid X_{i}\right) P\left(X_{i}\right)}=\frac{causal knowledge-prior knowledge}{prior knowledge}$ This is a good understanding of the expressed intention, which is the result $Y$ occurs because of the cause $X_i$ What is the probability of , that is, the probability inference of [effect → cause] (solving a crime is such a process). $P (Y)$ is a marginal probability comparison It is easy to understand that it is the probability of such an effect. If it is in two dimensions, according to the total probability formula, that is, (03) P ( Y ) = ∑ i P ( X i ) P ( Y ∣ X i ) P(Y)=\sum_iP(X_i)P(Y|X_i) $P (Y) = \sum_{i} P (X_{i}) P (Y ∣ X_{i})$ 。另外 $P\left(Y \mid X_{i}\right) P\left(X_{i}\right)$ , the meaning is relatively simple, it is based on $Y$ On the basis of event occurrence, at the same time $X_i$ The event also occurs. $P( X_{i})$ Approximate probability of first choice, $\mid X_{i})$ is called the likelihood probability. The prior probability is understood as the initial value, and the likelihood probability represents the modification of the prior, and then $P(X_i|Y)$ is called the posterior probability density. In addition, we usually record the denominator as
$\color{Green} \tag{08} \eta=\frac{1}{\sum_{i=1}^{n} P\left(Y \mid X_{i}\right) P\left(X_{i}\right)}$ 那么(07)式也会被写成 $\color{Green} \tag{09} P\left(X_{i} \mid Y\right)=\frac{P\left(Y \mid X_{i}\right) P\left(X_{i}\right)}{P(Y)}=\eta P\left(Y \mid X_{i}\right) P\left(X_{i}\right)$

2. Continuous derivation

is back to the continuous derivation. It doesn’t take much, and of course it will be relatively complicated. In the derivation process, the integration to infinite series (discrete) process is used, that is, continuous to discretization. Here is the first Derivation results, let’s discuss the derivation process: $y)=\frac{f_{X, Y}(x, y)}{f_{Y}(y)}=\frac{f_{Y \mid X}(y \mid x) f_{X}(x )}{\int_{-\infty}^{+\infty} f_{Y \mid X}(y \mid x) f_{X}(x) \mathrm{d} x}$ 根据前面知识，可以知道 $\color{Green} \tag{11} F_{X \mid Y}(x \mid y)=P(X \leq x \mid Y=y)=\int_{-\infty}^{x} f_{X \mid Y}(x \mid y) d x=\sum_{u=-\infty}^{x} P(X=u \mid Y=y)$ The above is a core step, which is $f_{X \mid Y}(x \mid y) d x$ as an infinitesimal quantity Treat, according to [Full Probability (Continuous)] (05) derivation, treat it as the area of a small interval, that is, $AND = y$ 条件下，对 $X = u$ Progress $(-\infty,x)$ . Now that it is discretized, of course we can apply the discrete formula (07), that is: $\color{Green} \tag{12} \sum_{u=-\infty}^{ x} P(X=u \mid Y=y)=\sum_{u=-\infty}^{x} \frac{P(Y=y \mid X=u) P(X=u)}{P (Y=y)}$ If you want to do something else, you can use the graphic graph to generate the graph :
$\color{Green} \tag{13} =\lim _{\varepsilon\rightarrow 0} \sum_{u=-\infty}^{x} \frac{P(y\ leq Y \leq y+\varepsilon \mid X=u) P(u \leq X \leq u+\varepsilon)}{P(y \leq Y \leq y+\varepsilon)}$ Why can equation (10) be converted into equation (11)? Yes Because when using discrete substitution, $X, Y$ all have a range, and now this range is considered to be infinite. Then apply Lagrange's mean value theorem: $\color{Green} \tag{14} = \lim _{\varepsilon \rightarrow 0} \sum_{u=-\infty}^{x} \frac{f_{Y \mid X}\left(\xi_{1} \mid u\right) \varepsilon \cdot f_{X}\left(\xi_{2}\right) \varepsilon}{f_{Y}\left(\xi_ {3}\right) \varepsilon} ,$ The main basis for the above is, as mentioned earlier, probability about random variables The reciprocal of is the probability density function. In the above formula $\xi_{1} \in(y, y+\varepsilon)$ ,、 $\xi_{2 } \in(u, u+\varepsilon)$ 、 $\xi_{3} \in(y, y+\varepsilon)$ 。会 $\varepsilon$ 趶向于无穷小， $\xi_{1}=u$ 、 $\xi_{2}=u$ 、 $\xi_{3}=y$ ; At the same time, the numerator and denominator are reduced by one $\varepsilon$ ，电影化简可得,：
$\color{Green} \tag{15} =\lim _{\varepsilon \rightarrow 0} \sum_{u=-\infty}^{x} \frac{f_{Y \mid X}(y \mid u) f_{X}(u)}{f_ {Y}(y)} \varepsilon=\int_{-\infty}^{x} \frac{f_{Y \mid X}(y \mid u) f_{X}(u)}{f_{Y} (y)} \mathrm d u$ cause $\varepsilon$ is a small amount， $\mathrm du$ is also an infinitesimal quantity, so the symbol above has changed. At the same time, $u$ conversion $x$ means, then we can get, then to summarize, it has been derived so far: $\mid y)=P(X \leq x \mid Y=y)=\int_{-\infty}^{x} \frac{f_{Y \mid X}(y \mid x) f_{X}(x)}{f_{Y}(y)} \mathrm d x$ According to the properties of the continuous random probability density function, we can get: $F_{X \mid Y}(x \mid y)=P(X \leq x \mid Y=y)=\int_{-\infty}^{x} f_{X \mid Y}(x \mid y) \mathrm d x$ Then according to (14) (15), we can get the continuous random variable Bayesian formula: $\color{Green} \tag{18} f_{X \mid Y}(x \mid y)=\frac{f_{Y \mid X}(y \mid x) f_{X}(x)}{f_{Y}(y)}=\eta f_{Y \mid X}(y \mid x) f_{X}(x)$ 此处， $f_{Y}(y)$ satisfies (defined by the conditional probability density): $f_{Y} (y) = \int_{- \infty}^{+ \infty} f (y, x) d x = \int_{- \infty}^{+ \infty} f_{Y ∣ X} (y ∣ x) f_{X} (x) d x (20)$ 所以 $\color{Green} \tag{21} \eta=\frac{1}{\int_{\mathbb{-\infty}}^{+\infty} f_{Y \mid X}(y \mid x) f_{X}(x) d x}$ is similar to discrete, $f_X(x)$ Name first approximate probability density, $f_{Y|X}(y|x)$ nominal approximate probability density, $f_{X|Y}(x|y)$ is called the posterior probability density function. The likelihood probability density function can modify the prior probability density function to obtain the posterior probability density function.

4. Formula record

1. Normal distribution density function:

$\color{Green} \tag{22} f(x)=\frac{ 1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}~~~~~~~~~ ~~X \sim N\left(\mu, \sigma^{2}\right)$ onwards $\mu$ Display uniformity, $\sigma$ Display standard difference.

2. Bayesian formula of normal distribution:

$\tag{23} Prior probability density function: ~f_X(x)=X \sim N\left(\mu_1, \sigma_1^{2}\right)=\frac{1}{ \sigma_1 \sqrt{2 \ pi}} e^{-\frac{(x-\mu_1)^{2}}{2 \sigma_1^{2}}}$ $~f_{Y|X}(y|x)=X \sim N\left(\mu_2, \sigma_2^{2}\right)=\frac{1}{\sigma_2\sqrt{2\pi}} e ^{-\frac{(x-\mu_2)^{2}}{2\sigma_2^{2}}}$ $\color{Green} \tag{25} f_{X \mid Y}(x \mid y)=\frac{f_{X, Y}(x, y)}{f_{Y}(y)}=\frac{f_{Y \mid X}(y \mid x) f_{X}(x)}{\int_{-\infty}^{+\infty} f_{Y \mid X}(y \mid x) f_{X}(x) \mathrm{d} x}=\eta f_{Y \mid X}(y \mid x) f_{X}(x)$ $\color{Green } \tag{26} also define the boundary value:~f_{X|Y}(x|y)=N\left(\frac{\sigma_{1}^{2}}{\sigma_{1}^2 +\sigma_{2}^{2}} \mu_{2}+\frac{\sigma_{2}^{2}}{\sigma_{1}^{2}+\sigma_{2}^{2} } \mu_{1} , \frac{\sigma_{1}^{2} \sigma_{2}^{2}}{\sigma_{1}^{2}+\sigma_{2}^{2}} \right)$ From this formula, it can be clearly known that after the likelihood probability density function After correction, the standard deviation (error) of the prior probability density function can be reduced. Because $\frac{\sigma_{1}^{2} \sigma_{2}^{2}}{\sigma_{1 }^{2}+\sigma_{2}^{2}}$ must be less than $\sigma_{1}^{2}$ 与 $\sigma_{2}^{2}$ 。