Table of contents
The Essential Connotation of Bayesian Formula
Introduction
Naive Bayesian classification algorithm is one of the most classic algorithms in machine learning, which is a supervised learning algorithm. Its theoretical basis is "Bayes' theorem", which was proposed by the famous British mathematician Thomas Bayes. Bayes' theorem is realized based on relevant knowledge of statistics and probability theory. Bayesian classification algorithm has a wide range of uses, such as widely used in sentiment classification, text classification and other classification tasks.
basic concept
In order to better understand the Bayesian classification algorithm, understand several basic concepts before theoretical derivation.
Prior probability: the probability based on empirical knowledge.
Posterior probability : the probability that an event occurs due to certain factors.
Conditional probability formula : Assuming events A and B, the probability of occurrence under the condition of known B is P(A|B), and the simultaneous occurrence of A and B events is recorded as
P(A, B), has the following conditional probability formula:
Joint probability: Joint probability refers to the probability that contains multiple conditions and all conditions are true at the same time, recorded as P(X=a,Y=b) or P(a,b), and some books are also used to record as P( ab), but this notation is not used by individuals, so the comma-separated notation is used below.
Joint probability formula:
(1) B1, B2.... are mutually exclusive, that is, Bi ∩ Bj = ∅ , i≠j, i, j=1, 2,...., and P(Bi)>0, i=1 ,2,....;
(2) B1∪B2∪....=Ω, then the event group B1,B2,...is said to be a division of the sample space Ω. Let B1,B2,... be a division of the sample space Ω, and A be any event, then:
The above formula is the total probability formula.
Bayes theorem
The discoverer of Bayes' theorem, Thomas Bayes, put forward a very interesting hypothesis: "If there are 10 balls in a bag, they are black balls and white balls, but we don't know what the ratio between them is. , now, can you judge the ratio of black and white balls in the bag just by the color of the balls you draw?"
The above problem may conflict with the probability we accepted in high school, because the probability problem you are exposed to may be like this: "There are 10 balls in a bag, 4 black balls and 6 white balls. If you randomly Grab a ball, what is the probability that it is black?" The answer is, of course, 0.4. This problem is very simple, because we know the ratio of black balls and white balls in the bag in advance, so it is easy to calculate the probability of touching a ball, but in some complicated cases, we cannot know the "proportion". Bayesian questions.
There are two big schools of thought in statistics: one is the "frequency" school and the other is the "Bayesian" school. They all have their own huge knowledge systems, and "Bayesian" mainly uses the "correlation" the word "sex". Let's describe "Bayes' theorem" in an easy-to-understand way: Generally, the probability of event A under the condition of event B occurring is not the same as that of event B under the condition of event A occurring, but they both There is a certain correlation between them and has the following formula (called "Bayesian formula"):
The Essential Connotation of Bayesian Formula
From cause to effect and from effect to cause
In reality, we can regard event A as the result, and events B1, B2,..., Bn as various causes leading to this result. Then, the full probability formula we introduced
P(A)=P(B1)P(A|B1)+P(B2)P(A|B2)+...+P(Bn)P(A|Bn)
It is to deduce the probability of the result event from various reasons, from cause to effect.
However, there is actually an important application scenario: we often observe a certain phenomenon in our daily life, and then deduce the probability of various causes of this phenomenon. To put it simply, it is to infer the cause from the effect.
the code
from sklearn.datasets import fetch_20newsgroups # 从sklearn.datasets里导入新闻数据抓取器 fetch_20newsgroups
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer # 从sklearn.feature_extraction.text里导入文本特征向量化模块
from sklearn.naive_bayes import MultinomialNB # 从sklean.naive_bayes里导入朴素贝叶斯模型
from sklearn.metrics import classification_report
#1.数据获取
news = fetch_20newsgroups(subset='all')
print(len(news.data)) # 输出数据的条数：18846
#2.数据预处理：训练集和测试集分割，文本特征向量化
X_train,X_test,y_train,y_test = train_test_split(news.data,news.target,test_size=0.25,random_state=33) # 随机采样25%的数据样本作为测试集
#print X_train[0] #查看训练样本
#print y_train[0:100] #查看标签
#文本特征向量化
vec = CountVectorizer()
X_train = vec.fit_transform(X_train)
X_test = vec.transform(X_test)
#3.使用朴素贝叶斯进行训练
mnb = MultinomialNB() # 使用默认配置初始化朴素贝叶斯
mnb.fit(X_train,y_train) # 利用训练数据对模型参数进行估计
y_predict = mnb.predict(X_test) # 对参数进行预测
#4.获取结果报告
print('The Accuracy of Naive Bayes Classifier is:', mnb.score(X_test,y_test))
print(classification_report(y_test, y_predict, target_names = news.target_names))
Downloading 20news dataset. This may take a few minutes.
Downloading dataset from https://ndownloader.figshare.com/files/5975967 (14 MB)
18846
The Accuracy of Naive Bayes Classifier is: 0.8397707979626485
precision recall f1-score support
alt.atheism 0.86 0.86 0.86 201
comp.graphics 0.59 0.86 0.70 250
comp.os.ms-windows.misc 0.89 0.10 0.17 248
comp.sys.ibm.pc.hardware 0.60 0.88 0.72 240
comp.sys.mac.hardware 0.93 0.78 0.85 242
comp.windows.x 0.82 0.84 0.83 263
misc.forsale 0.91 0.70 0.79 257
rec.autos 0.89 0.89 0.89 238
rec.motorcycles 0.98 0.92 0.95 276
rec.sport.baseball 0.98 0.91 0.95 251
rec.sport.hockey 0.93 0.99 0.96 233
sci.crypt 0.86 0.98 0.91 238
sci.electronics 0.85 0.88 0.86 249
sci.med 0.92 0.94 0.93 245
sci.space 0.89 0.96 0.92 221
soc.religion.christian 0.78 0.96 0.86 232
talk.politics.guns 0.88 0.96 0.92 251
talk.politics.mideast 0.90 0.98 0.94 231
talk.politics.misc 0.79 0.89 0.84 188
talk.religion.misc 0.93 0.44 0.60 158
micro avg 0.84 0.84 0.84 4712
macro avg 0.86 0.84 0.82 4712
weighted avg 0.86 0.84 0.82 4712
Thomas Bayes
Bayesian (about 1701-1761) Thomas Bayes, British mathematician. Born in London about 1701, he was a priest. He became a member of the Royal Society in 1742. Died April 7, 1761. Bayesian mathematics mainly studies probability theory . He first applied the inductive reasoning method to the basic theory of probability theory, and created the Bayesian statistical theory, and made contributions to statistical decision functions, statistical inference, and statistical estimation. After his death, Richard Price sent his book "An essay towards solving a problem in the doctrine of chances" to the Royal Society in 1763. Probability theory and mathematical statistics had an important impact. Another book by Bayes, "Introduction to the Doctrine of Chance", was published in 1758. Many of the terms used by Bayesian are still used today.