Reproduced, please indicate place of origin, the article starts:
https://www.zhoulujun.cn/html/theory/math/2017_0913_8050.html
1 Bayesian approach
For a long time, people on the probability that an event occurs or does not occur, only a fixed 0 and 1, that either occurs or does not occur, never to consider the likelihood that something will happen much, the probability of occurrence is not how much. And although the probability is unknown, but at least is a definite value. For example, if a question asked people at that time: "? There is a bag which contains a number of white balls and black balls, the probability of Will made white balls from the bag how much" they will no-brainer, it will immediately tell you, remove the white ball the probability is 1/2, or get to the white ball, or take less than the white ball, that can only have a value of θ, and no matter how many times you take, get the white ball probability θ is always 1/2, that is, not with observation X changes varies.
This frequency school of thought long dominated people's ideas, however:
Suppose we have the following seven balls in the A, B two boxes, if we just take a ball, known to take the ball from the B box, then the ball is the probability that the white ball is how much? Or asked to remove the balls are white, then the probability is taken from B box is how much? This problem is not solved, until a man named Thomas Bayes characters appear.
1.1 Bayesian approach proposed
Bayes Thomas Bayes Thomas (1702-1763) was still alive, was not as well known, very few published papers or books published, and then academics who rarely communicate with words now, Bayes is a living folk academic "Cock wire" can be the "Cock wire" eventually published an article entitled " an Essay towards Solving a problem in at the Doctrine of Chances ", translation is: solving a problem of opportunity theory . You may think I have to say: the publication of this paper is randomly generated sensational effect, thereby laying the Bayesian position in the academic history.
In fact, after the paper was published, did not produce much influence at the time, in the 20th century, the paper gradually valued by the people. In this regard, the Van Gogh similar repeat itself, worthless paintings during his lifetime, after the death priceless.
Back to the example above: "There is a bag which contains a number of white balls and black balls, will get the white ball from bag probability θ is how much?" Bayesian probability to get the white ball is indeterminate value, because it contains ingredients opportunities. For example, a business friend, you obviously know the results of entrepreneurship on the two kinds, namely either succeed or fail, but you still can not help but to estimate his chances of success and how much? If you are on a better understanding of his man, but there are ways, clear thinking, perseverance, and can unite people around you will not help him estimate the chances of success and may be more than 80%. This differs from the beginning of "black and white, i.e. a non-zero" way of thinking, it is a Bayesian way of thinking.
Before proceeding further explain the Bayesian method, briefly summarize the frequency of Bayeux Spirax School and their different ways of thinking:
Send the required frequency parameter θ inferred seen as fixed unknown constant, that although the probability of θ is unknown, but at least one value is determined, at the same time, X is a random sample, the frequency send key research sample space, large probability is part of the calculation for the distribution of sample X;
The Bayeux Spirax view is diametrically opposed, they believe parameter θ is a random variable, while the sample X is fixed, because the sample is fixed, so they focus on is the distribution parameter θ.
In contrast, the frequency of the school of thought is easy to understand, it focuses on the Bayeux Spirax views below.
Since the θ Bayeux Spirax seen as a random variable, so to calculate the distribution of θ, we have to know in advance θ unconditional distribution, that is, before there is a sample (or prior to the observed X), θ has a kind of distribution it?
For example, the pool table to throw a ball, where the ball will fall off it? If it is impartial to throw the ball, then the ball falls on a billiard table in any position have the same opportunities that the probability of a billiard table position of the ball falls to obey a uniform distribution . The basic premise distribution properties belonging to this set is called before the experiment prior distribution, distribution or unconditional.
So far, Bayesian and Bayesian Spirax proposed a fixed mode of thinking:
Prior distribution π (θ) + After the sample information χ⇒ posterior distribution π (θ | x)
The above means that mode of thinking, a new sample information observed in the previous amendments on people's perception of things. In other words, before getting a new sample information, awareness of people on the a priori distribution π (θ), χ after obtaining the new sample information, people's awareness of θ is π (θ | x).
Then the posterior distribution π (θ | x) generally considered to be conditional distribution of θ in the case of a given sample χ, and the maximum value is called the maximum a posteriori estimation θMD, similar to the classic statistics of ML However, it estimates .
Taken together, it is like when human nature is only the beginning for very little prior knowledge, but with the constant is observed experimentally obtained more samples, the results, making it more tangible laws of nature thorough. Therefore, the Bayesian approach is in line with thinking people's daily life, but also in line with the laws of nature awareness, through constant development, and ultimately occupy half of the field of statistics, with the classic statistics rival.
In addition, the proposed addition to the above Bayesian thinking patterns, but also proposed a world-famous Bayes' theorem.
1.2 Bayes Theorem
Before leads to Bayes' theorem, first learn a few definitions:
Marginal probability (also known as the a priori probability ): the probability of an event occurring. Marginal probability is obtained in this way: In the joint probability, the final result of those events combined into unwanted by their total probability, and eliminate them (discrete random variables summed to give the total probability of continuous random variables with points obtain the probability of the whole), which is referred marginalization (marginalization), such as marginal probability of a is represented as P (a), B is represented by an edge probability P (B).
Joint probability represents the probability of two events occur together. The joint probability of A and B is expressed as P (A∩B) or P (A, B).
Conditional probability ( also known as the posterior probability ) : A probability of occurrence of an event has occurred under conditions in another event B. Expressed as a conditional probability P (A | B), is read as "the probability of B at condition A".
Next, consider the question: P | possibility (A B) in the case of occurrence of A B occurs.
First, prior to the occurrence of event B, we have to occurrence of an event A is the probability of determining a basic, known as the prior probability of A, denoted by P (A);
Secondly, after the event B happens, we re-evaluate the probability of occurrence of event A, called the posterior probability of A with P (A | B) represented;
Similarly, before the event A occurs, we have the occurrence of event B determines a basic probability, referred to as prior probability B is denoted by P (B);
Similarly, after the event A occurs, we re-evaluate the probability of occurrence of event B, called the posterior probability of B with P (B | A) represented.
Bayes' theorem is based on the Bayesian formula:
P(A|B)=P(B|A)P(A)/P(B)
Derivation of the above equation is actually very simple, that is out of the conditional probability.
According to the definition of conditional probability, the probability of event A occurs at event B is
P(A|B)=P(A∩B)/P(B)
Similarly, the probability, under the conditions of event A event B occurs
P(B|A)=P(A∩B)/P(A)
Consolidation and merger of the two equations, we can obtain:
P(A|B)P(B)=P(A∩B)=P(B|A)P(A)
Next, on both sides of the same formula dividing P (B), if P (B) is non-zero, we can get the expression Bayesian formula:
P(A|B)=P(B|A)*P(A)/P(B)
I was watching " comes from a Bayesian approach Bayesian networks ," the time to see here, in fact, already reeling up.
P (A | B) and P | regular people (B A) such confusion, Chen @ JobHunting gives a key point to understand the distinction between the laws and phenomena, is to A as "law ", B as" phenomenon ", so as Bayesian formula:
In Chen "It's understanding Bayesian formula you" and "another of life Bayesian application" gives several examples of user-friendly, not repeat them here.
Reference article:
Speaking from a Bayesian network Bayesian method
Chapter 1 Bayesian inference thinking
Full stack necessary Bayesian approach
Bayesian formula really understand it?
Understand the total probability formula and Bayes' formula
Bayesian inference and Internet applications (a): Theorem Introduction
Machine Learning (a) - Discussion and Bayesian MCMC (recommended reading)