PT@Total probability formula and Bayesian formula@Posterior probability and belief measure

abstract

• Total probability formula and bayes formula and their applications
• Posterior probabilities and belief measures

Complete event group (partition)

• Let the finite set I = { 1 , ⋯ , n } I=\{1,\cdots,n\}I={ 1,⋯,n } ;Trial$The\; sample\; space\; of\; E$ is$Oh$
• 若 { B i ; i ∈ I } \{B_i;i\in{I}\} { Bi​;i∈I } satisfies:
  • ${\bigcup }_{i=1}^n{B}_i=Oh$
  • $B_i{B}_j=\varnothing ;i\ne j$
• 则称$\{B_i;i\in I\}$为Ω \OmegaAcomplete event group$of\; \Omega$ , also calledpartition

basic properties

• Complete case set { B i ; i ∈ I } \{B_i;i\in{I}\}{ Bi​;i∈I } , any sample point of the test (any test result) belongs to and only belongs to a certain$B_i$

total probability formula

• Let the sample space of experiment E be $S$,$A$ -$E$ 's event

• {   B i ∣ i ∈ I   } \set{B_i|i\in I} { Bi​∣i∈I} is a$S$的划分,$P(B_i)>0,i\in I$,则$P(A)={\sum }_{i=1}^{n}P(A|B_i)P(B_i)$

  • 那么$P(A)=\sum _{i\in I}P(A|B_i\left)P\right(B_i)$

• prove:

  • Abundant $A{B}_i\subset B_i$,又$B_i{B}_j=\varnothing$,所以$(A{B}_i)(A{B}_j)=A(B_i{B}_j)=A\varnothing =\varnothing$ ,$(i\ne j)$

  • $P(A|B_i)P(B_i)=P(A{B}_i)$

  • Proof 1:

    • $\sum _{i\in I}P(A|B_i\left)P\right(B_i)=\sum _{i\in I}P(A{B}_i)$
      • $=P(A{B}_1\cup A{B}_2\cup \cdots \cup A{B}_n)$
      • $=P\left(A\cap \left(\bigcup _{i\in I}{B}_i\right)\right)$
      • $=P\left(A\Omega \right)$
      • $=P(A)$

  • Proof 2:

    • $A=AS=A(B_1\cup \cdots \cup B_n)$=$A{B}_1\cup \cdots \cup A{B}_n$
    • $P(A)$=$P(A{B}_1\cup \cdots \cup A{B}_n)$=${\sum }_{i=1}^{n}P(A{B}_i)$=$\sum _{i\in I}P(A|B_i)P(B_i)$

example

• A box containing 20 balls

  • The probabilities of containing 0, 1, and 2 defective products are 0.8, 0.1, and 0.1 respectively.

    • Record: $A_i$={The number of defective products contained in the box is $i$ }
      • $P(A_0)=0.8$
      • $P\left(A_1\right)=P\left(A_2\right)=0.1$

  • Remember $B$ ={The 4 products selected are allgenuine}

• Then event $The\; probability\; of\; B$ ?

  • A 0 , A 1 , A 2 A_0,A_1, A_2$A_0,A_1,A_2$It constitutes a division of the sample space of test E {observe the number of genuine balls in all the boxes, and the sample space is {0,1,2}}

  • And event BBTest FFconducted by$B$$F$ {Observe the number of genuine products taken out of the 4 samples, the sample space is {0,1,2}}

  • It is easy to calculate the following conditional probability based on the classical summary formula (because the sample space at this time is known) (here we do not use the conditional probability formula to expand the calculation, which will circle back)

    • $P(B|A_0)=\frac{\left(\genfrac{}{}{0ex}{}{20}{4}\right)}{\left(\genfrac{}{}{0ex}{}{20}{4}\right)}=1$

    • $P(B|A_1)=\frac{\left(\genfrac{}{}{0ex}{}{19}{4}\right)}{\left(\genfrac{}{}{0ex}{}{20}{4}\right)}=\frac{(19\ast 18\ast 17\ast 16)\ast (4\ast 3\ast 2\ast 1)}{(20\ast 19\ast 18\ast 17)\ast (4\ast 3\ast 2\ast 1)}=\frac{4}{5}$

      • Among them, ${Oh}_{A_1}=Oh$

    • $P(B|A_2)=\frac{\left(\genfrac{}{}{0ex}{}{18}{4}\right)}{\left(\genfrac{}{}{0ex}{}{20}{4}\right)}=\frac{12}{19}$

  • According to the full probability formula $P(B)=\sum _{i=1}^{3}P(B|A_i)P(A_i)=0.943$

  • Note: Test $The\; sample\; space\; of\; F$ happens to overlap with the sample space of E, but if there are more than 4 defective products in the box, thenFFThe sample space of$F$$0,1,2,3,4$ is included inEEof the sample space of$E$

Bayesian formula

• The Bayes formula is based on the above-mentioned total probability formula. It is a more comprehensive formula, but the principle is simple.
• Let the sample space of experiment E be $S$,$A$ -$E$ 's event
• {   B i ∣ i ∈ I   } \set{B_i|i\in I} { Bi​∣i∈I} is a$S$的划分,$P(A)>0,P(B_i)>0,i\in I$ , 则
  • $P(B_i|A)$=$\frac{P(A{B}_i)}{P\left(A\right)}$=$\frac{P(A|B_i\left)P\right(B_i)}{{\sum }_{j=1}^{n}P(A|B_j)P(B_j)}$;$i=1,\cdots ,n$ ;This formula is bayes formula
• Proof: Based on conditional probability, multiplication theorem, total probability formula, Bayes' formula is obviously established
  • $P(B_i|A)$=$\frac{P(A{B}_i)}{P(A)}$
  • $P(A{B}_i)$=$P(A|B_i\left)P\right(B_i)$
    • Note that in the bayes formula, what we require is $P(B_i|A)$ , so$P(A{B}_i)$=$P(B_i|A\left)P\right(A)$ is not appropriate, otherwise the formula cannot be calculated effectively
    • And therefore we should use $P(A{B}_i)$=$P(A|B_i)P(B_i)$ next calculation
  • $P(A)={\sum }_{i=1}^{n}P(A|B_i\left)P\right(B_i)$

example

• Source of defective products: Suppose a batch of parts comes from three suppliers

  • supplier	Defective rate	Purchase share
    1	0.02	0.15
    2	0.01	0.80
    3	0.03	0.05

• Test content: Extract one piece from the parts

  • A={The product received is defective}
  • $B_i$={Defective parts come from $i$ manufacturers}
    • $P(B_1)$=$0.15$;$P(A|B_1)$=0.02
    • $P(B_2)$=0.80;$P(A|B_2)$=0.01
    • $P(B_3)$=0.05;$P(A|B_3)$=0.03

• Find the probability that the sample is defective:

  • Total approximation formula: $P(A)=\sum _{i=1}^{3}P(A|B_i)P(B_i)$=$0.02\ast 0.15+0.01\ast 0.80+0.03\ast 0.05=0.0125$

• If you take out one piece and find that it is defective, then it comes from the manufacturer iiWhat is the probability of$i$$(i=1,2,3)$

  • According to Bayesian formula:
    • $P(B_i|A)=\frac{P(B_{i}A)}{P(A)}=\frac{P(A|B_i\left)P\right(B_i)}{P\left(A\right)}$
  • It can be calculated respectively:
    • $P(B_1|A)=\frac{0.02\ast 0.15}{0.0125}=0.24$
    • $P(B_2|A)$=$0.64$
    • $P(b_3|A)$=$0.12$

Common forms for opposing events

• The total probability formula and beyes formula are at $n=$When$2$$B,\overline{B}$constitutes a partition of the sample space) most commonly used
• At this time there are:
  • $P(A)=P(AB)+P(A\overline{B})$=$P(A|B)P(B)+P(A|\overline{B})P(\overline{B})$
  • $P(B|A)=\frac{P(AB)}{P(A)}$=$\frac{P(A|B)P(B)}{P(A|B)P(B)+P(A|\overline{B})P(\overline{B})}$

example

• Suppose there are 2 red balls and 1 white ball in the first box; the second box has half red balls and half white balls.
• If you first take one ball from each of the two boxes to make a pair, and then choose one ball from the two boxes,
  1. Event $C$ : What is the probability of getting a red ball?
  2. Event CC occurredUnder the condition of$C$$D$ : What is the probability that this red ball comes from the first box?
• untie
• (1)
  • method 1:
    • order $A_i$:{The final selected ball comes from $i$ each box},$i=1,2$
    • From the total probability formula, we know that $P(C)={\sum }_{i=1}^{2}P(C|A_i)P(A_i)$=$\frac{2}{3}\times \frac{1}{2}+\frac{1}{2}\times \frac{1}{2}$=$\frac{7}{12}$
  • Method 2:
    • order $A_i$:{from iiThe red ball drawn from$i$$i=1,2$
    • 令$B_1=A_1{A}_2$;$B_2=\overline{A_1}{A}_2$;$B_3=A_1\overline{A_2}$;$B_4=\overline{A_1}\overline{A_2}$
      • $P(B_1)=P(A_1)P(A_2)=\frac{2}{3}\times \frac{1}{2}$=$\frac{1}{3}$
      • Similarly, $P(B_2)$=$\frac{1}{3}\times \frac{1}{2}$=$\frac{1}{6}$;$P(B_3)$=$\frac{2}{3}\times \frac{1}{2}$=$\frac{1}{3}$;$P(B_4)=\frac{1}{3}\times \frac{1}{2}$=$\frac{1}{6}$
    • From the total probability formula: $P(C)$=${\sum }_{i=1}^{4}P(C|B_i)P(B_i)$=$1\times \frac{1}{3}+\frac{1}{2}\times \frac{1}{6}+\frac{1}{2}\times \frac{1}{3}+0\times \frac{1}{6}$=$\frac{7}{12}$
• (2)
  • order $A_i$:{from iiThe red ball drawn from$i$$i=1,2$
  • method 1:
    • $P(A_1|C)$=$\frac{P(C|A_1)P(A_1)}{P(C)}$
      • $P(C|A_1)$=$P(C|A_1{A}_2\left)P\right(A_1{A}_2)+P(C|A_1\overline{A_2})P(A_1\overline{A})$=$1\times \frac{1}{2}+\frac{1}{2}\times \frac{1}{2}$=$\frac{3}{4}$
    • $P(A_1|C)$=$\frac{\frac{3}{4}\times \frac{2}{3}}{7/12}$=$\frac{6}{7}$
  • Method 2:
    • $A_1=A_1{A}_2+A_1\overline{A_2}$
    • $P(A_1|C)$=$P([A_1{A}_2+A_1\overline{A_2}]|C)$=$P(A_1{A}_2|C)P(A_1\overline{A_2}|C)$=$P(A_1{A}_{2}C)/P(B)+P\left(A_1\overline{A_2}C\right)/P\left(B\right)$
      • =$P(C|A_1{A}_2)P(A_1{A}_2)/P(B)$+$P(C|A_1\overline{A_2}\left)P\right(A_1\overline{A_2})/P(B)$
      • =$\frac{6}{7}$

example

• There are two boxes, the first box contains $n$ pieces of first-class goods, the second box has$m$ products including only 1 first-class product
• Choose one of the two boxes, and then take out one product from the box one after another
  1. What is the probability that the part taken out the first time is a first-class product? What is the probability that the part taken out the second time is a first-class product?
  2. If it is known that the first item taken out is a first-class product, what is the probability that the second item is still a first-class product?
• (1)
  • order $A_i$:The selected box is the $i$ boxes (or$A$ : The first box is selected,$\overline{A}$:The second box is selected)
  • Order $B_i$:Part iiThe item taken out$i$$i=1,2$
    • $P(B_1)$=$\frac{1}{2}\times 1+\frac{1}{2}\times \frac{1}{m}$=$\frac{m+1}{2m\_}$
    • $P(B_2)$=$P(B_2|A_1)P(A_1)+P(B_2|A_2)P\left(A_2\right)$
      • =$P(B_2|A_1{B}_1\left)P\right(A_1{B}_1)+P(B_2|A_1\overline{B_1})P(A_1\overline{B_1})$+$P(B_2|A_2{B}_1\left)P\right(A_2{B}_1)+P(B_2|A_2\overline{B_1})P\left(A_2\overline{B_1}\right)$
      • $P(B_2|A_1\left)P\right(A_1)$=$1\times \frac{1}{2}$
      • $P(B_2|A_2\left)P\right(A_2)$=$P(B_2|A_2{B}_1)P(A_2{B}_1)+P(B_2|A_2\overline{B_1})P(A_2\overline{B_1})$=$0+P(B_2|A_2\overline{B_1})P(\overline{B_1}|A_2)P(A_2)$=$\frac{1}{2m\_}$
      • $P(B_2)$=$\frac{1}{2}+\frac{1}{2m\_}$=$\frac{1+m}{2m\_}$
    • In fact, if the box selected is the first box, then the first-class product will be taken out both times; if the box selected is the second box, based on the drawing, the first-class product and the first-class product will be drawn in the first draw. The probability of winning the first-class product in the second draw is $\frac{1}{m}$, so the first-class product drawn for the first time and the second time are always equal.
• (2)
  • method 1:
    • $P(B_2|B_1)$=$\frac{P(B_1|B_2)P(B_2)}{P(B_1)}$
      • $P(B_1|B_2)$=$P(B_1|A_1{B}_2)P(A_1{B}_2)$+$P(B_1|A_2{B}_2\left)P\right(A_2{B}_2)$=$P(B_1|A_1{B}_2)P(B_2|A_1)P(A_1)$+$P(B_1|A_2{B}_2\left)P\right(B_2|A_2)P(B_2)$=$1\times 1\times \frac{1}{2}+0$=$\frac{1}{2}$
      • $P(B_2|B_1)=\frac{1}{2}$
    • This method uses the Bayes formula for direct calculation, but you will find that it is not too straightforward to calculate the two probabilities of the molecule.
    • The bayes formula is a model for solving this kind of problem, but it is not necessarily the simplest. Constructing more appropriate sample spaces and events may simplify the solution process.
  • Method 2:
    • Let event $D$ : B 1 B_1occurs$B_1$B 2 B_2 occurs under the conditions$B_2$
    • Then from the total probability formula: $P(D)$=$P(D|A_1)P(A_1)+P(D|A_2)P(A_2)$=$1\times \frac{1}{2}+0\times \frac{1}{2}$=$\frac{1}{2}$
    • In other words, event DDThe occurrence of$D$$A_1$Or $A_2$occurs, $A_1$If it happens, $D$ must happen, otherwise$D$ will definitely not happen, this$m,It$ doesn't matter
    • It can be seen that method 2 is much simpler and more direct
  • Wrong solution:
    • $P(B_2|B_1)$=$\frac{P(B_1{B}_2)}{P(B_1)}$
      • =$\frac{P(B_1{B}_2|A_1)P(A_1)+P(B_1{B}_2|A_2)P\left(A_2\right)}{P\left(B_1\right)}$=$\frac{m}{m+1}$
      • This result is about $The\; formula\; of\; m$ , when$When\; m$ is very large,$P(B_2|B_1)$ will almost certainly happen, which is obviously wrong
  • Sample space analysis:
    • S={ $(a_1,a_2),\cdots ,(a_1,a_n),\cdots ,(a_{n-1}{a}_n)$,$(b_1,b_2),\cdots ,(b_1,b_m),\cdots ,(b_{m-1}{b}_m)$}
      • 其中$a_i,i=1,\cdots ,n$ represents the parts in the first box;$b_j,j=1,\cdots ,m$ represents the parts in the second box
      • $(a_{i_1},a_{i_2}),i_1,i_2=1,\cdots ,n$ total$A_n^2$个样本点;$(b_{i_1},b_{i_2}),i_1,i_2=1,\cdots ,m$ has a total of$A_m^2$sample points;
      • These two types of sample points do not occur with equal probability, but similar sample points occur with equal probability, respectively $\frac{1}{2}\times \frac{1}{A_n^2}$;$\frac{1}{2}\times \frac{1}{A_m^2}$
      • But these are not important for this question
    • Let event $D_1$:$B_1,B_2$It all happened. It can be seen from the sample space that $D=D_1=B_1{B}_2$, so $P(D)$=$P(D_1)$=$\frac{1}{2}$
• Note: If the mm in the second box$m$ samples contain$k$ first-class goods, then the event$What\; is\; the\; probability\; of\; D$ ?
  • $P(D)=P(D|A_1)P(A_1)+P(D|A_2)P(A_2)$=$1\times \frac{1}{2}+\frac{k-1}{m-1}\times \frac{1}{2}$=$\frac{m+k-2}{2(m-1)}$

example

• Suppose there are registration forms for 3 regions with 10, 15, and 25 candidates each. The registration forms for girls are $3,7,5$ servings
• Now randomly select a region, and then take out 2 copies from the registration form in that region.
  1. What is the probability that the first table is a girl?
  2. It is known that the second part is a boy's table, what is the probability of the first part being a girl's table?
• order $A_i$: When the selected area iiWard$i$$i=1,2,3$;$B_j$: $j$ registration forms are for girls,$j=1,2$
• The ratio $P(A_1)=P\left(A_2\right)=P\left(A_3\right)=\frac{1}{3}$
  • $P(B_1|A_1)=\frac{3}{10}$;$P(B_1|A_2)=\frac{7}{15}$;$P(B_1|A_3)=\frac{5}{25}$
• (1): From the total probability formula: $P(B_1)$=${\sum }_{i=1}^{3}P(B_1|A_i)P(A_i)$=$\frac{29}{90}$
• (2): Let $D$:$B_2$When it happens $B_1$occur
  • $P(D)={\sum }_{i=1}^{3}P(D|A_i)P(A_i)$=$\frac{3}{9}\times \frac{1}{3}+\frac{3}{9}\times \frac{7}{14}$+$\frac{3}{9}\times \frac{5}{24}$=$\frac{25}{72}$
  • Method 2: $P(D)$=${\sum }_{i=1}^{3}P(B_1|A_i)P(A_i)$=$\frac{25}{72}$

Prior probability and posterior probability

example

• Machine and product qualification rate issues

• Assume that when the machine is normal, the pass rate of the products produced is 0.9, otherwise the pass rate is 0.3

• If the machine is turned on, the normal probability is 0.75 ( prior probability )

• On a certain day, the first product of the machine is qualified. What is the probability that the machine is normal?

• analyze:

  • A={The first product is qualified}
  • B={machine is normal}
  • The required probability expression is: $P(B|A)=?$

• According to the assumptions:

  • $P(A|B)=0.9;P(A|\overline{B})=0.3$
  • $P(B)=0.75,P(\overline{B})=0.25$
    • $B,\overline{B}$Constitutes a division of the sample space (that is, the machine is either normal or abnormal)
  • From the total probability formula $P(A)=P(A|B)P(B)+P(A|\overline{B})P(\overline{B})$=$0.9\ast 0.75+0.3\ast 0.25=0.75$
  • Then according to Bayes formula, $P(B|A)=\frac{P(A|B)P(B)}{P(A)}$=$\frac{0.9\ast 0.75}{0.75}=0.9$,
  • That is, under the condition that the first product is qualified, the posterior probability of the machine being normal is 0.9

• Posterior probability is a modification of prior probability

• The interpretation of posterior probability and prior probability is divided into two schools

  • Objectivist: All first products are qualified within 100 days, on average by 90 days the machine is normal
  • Subjective school: reflects people’s different subjective beliefs about the state of the machine before and after the experiment

Probability as a measure of people's beliefs about objective events

• Take Aesop’s fable The Boy Who Cried Wolf as an example.
• $A$ ={Child lies};$B$ = {Child is trustworthy}, assuming that atrustworthychild is relatively unlikely to lie, you might as well set this probability to 0.1, that is,$P(A|B)$ =0.1; conversely, the probability of an untrustworthy child telling a lie is 0.5, that is,$P(A|\overline{B})=0.5$
• Assume that the probability of a villager meeting a trustworthy child is 0.8, that is, $P(B)$=0.8
• Then the probability of the child saying something panic is $P(A)$=$P(A|B)P(B)+P(A|\overline{B})P(\overline{B})$=$0.1\times 0.8+0.5\times 0.2$=0.18
• Now assume that after a child tells a lie once, calculate the probability that the child is trustworthy according to the Bayes formula.
  • $P(B|A)$=$\frac{P(BA}{P(A)}$=$\frac{P(A|B)P(B)}{P(A|B)P(B)+P(A|\overline{B})P(\overline{B})}$=$\frac{0.1\times 0.8}{0.18}$=$\frac{4}{9}\approx 0.444$
  • That is, the posterior probability that the child is credible is reduced to 0.444
• If the child lies again, let $P(B)=0.444$ Calculate the posterior probability again as above:
  • $P(B|A)$=$\frac{0.444\times 0.1}{0.444\times 0.1+0.566\times 0.5}$=$0.138$
• It can be seen that after the child has behaved twice, the posterior probability of his/her credibility has been reduced to 0.138, giving the impression that he is almost an untrustworthy person.