Application of Naive Bass algorithm based on sklearn

1. Naive Bayes Algorithm

Naive Bayes method is a classification method based on Bayes' theorem and the assumption of independence of characteristic conditions. For a given data set, first learn the joint probability distribution of input and output based on the independent assumption of characteristic conditions; then, based on this model, for a given input x, use Bayes' theorem to find the output y with the largest posterior probability.
Naive Bayes is a typical generative learning method.

1.1 Learning and Classification of Naive Bayes Method

• basic method

• The meaning of maximizing posterior probability

1.2 Parameter estimation of Naive Bayes method

• Maximum likelihood estimation

• Learning and classification algorithms

  • Naive Bayes algorithm

1. Calculate prior probability and conditional probability

$$P(Y=c_k)=\frac{\sum _{i=1}^{N}I(y_i=c_k)}{N},k=1,2,...,K$$

$$P(X^{(j)}=a_{jl}|Y=c_k)=\frac{\sum _{i=1}^{N}I(x_i^{(j)}=a_{j}l,{and}_i=c_k)}{\sum _{i=1}^{N}I(y_i=c_k)}$$
$j=1,2,...,n;l=1,2,...,S^j;k=1,2,...,K$

2. For the given example $x=(x^{(1)},x^{(2)},...,x^{(n)}{)}^T$,计算：
$$P(Y=c_k)\prod _{j=1}^{n}P(X^{(j)}=x^{(j)}|Y=c_k)$$

$$k=1,2,...,K$$

3. Determine the class of the instance

$$and=arg\underset{c_k}{\max }P(Y=c_k)\prod _{j=1}^{n}P(X^{(j)}=x^{(j)}|Y=c_k)$$

2. Algorithm application

2.1 Train a classifier for continuous data

Problem description: The characteristics of the data are continuous (non-discrete values), train a naive Bayes classifier

solution:

from sklearn import datasets
from sklearn.naive_bayes import GaussianNB

#加载数据
iris = datasets.load_iris()
features = iris.data
target = iris.target
#创建高斯朴素贝叶斯对象
classifer = GaussianNB()
#训练模型
model = classifer.fit(features,target)

2.2 Train a classifier for discrete data and count data

Problem description: Given discrete data or count data, train a naive Bayes classifier

Solution: use a polynomial naive Bayes classifier

import numpy as np
from sklearn.naive_bayes import MultinomialNB
from sklearn.feature_extraction.text import CountVectorizer
#创建文本
text_data = np.array(['I love Brazil.Brazil!',
                      'Brazil is beat',
                      'Germany beats both'])
#创建词袋
count = CountVectorizer()
bag_of_words = count.fit_transform(text_data)
#创建特征矩阵
features = bag_of_words.toarray()
#创建目标向量
target = np.array([0,0,1])
#给定每个分类的先验概率，创建一个多项式朴素贝叶斯对象
classifer = MultinomialNB(class_prior=[0.25,0.5])
#训练模型
model = classifer.fit(features,target)

The working principle of the polynomial naive Bayes classifier is similar to that of the Gaussian naive Bayes classifier, but the characteristic of the sample data is to obey the polynomial distribution. In practical applications, this means that this classifier is mainly used when the data is discrete (for example, the rating of a movie, from 1 to 5). The most common scenario where a polynomial naive Bayes classifier is used is text classification, using bag-of-words or TF-IDF methods.

2.3 Train a naive Bayes classifier for data with binary features

Problem description: Some data with binary features need to train a naive Bayes classifier

Solution: Use Bernoulli Naive Bayes Classifier

import numpy as np
from sklearn.naive_bayes import BernoulliNB
#创建三个二元特征
features = np.random.randint(2,size=(100,3))
#创建一个二元目标向量
target = np.random.randint(2,size=(100,1)).ravel()
#给定每个分类的先验概率，创建一个多项式伯努利朴素贝叶斯对象
classifer = BernoulliNB(class_prior=[0.25,0.5])
#训练模型
model = classifer.fit(features,target)

Bernoulli's naive Bayes classifier assumes that all features are binary classification, so that they have only two values ​​(for example, nominal classification features that are one-hot encoded). Bernoulli Naive Bayes is often used for text classification. At this time, the feature matrix indicates whether a word appears in a document.

2.4 Calibration prediction probability

Problem description: Calibrate the classification probabilities obtained by the Naive Bayes classifier so that they can be interpreted

Solution: use CalibratedClassifierCV

from sklearn import datasets
from sklearn.naive_bayes import GaussianNB
from sklearn.calibration import CalibratedClassifierCV
#加载数据
iris = datasets.load_iris()
features = iris.data
target = iris.target
#创建高斯朴素贝叶斯对象
classifer = GaussianNB()
#创建使用sigmoid校准调校过的交叉验证模型
classifer_sigmoid = CalibratedClassifierCV(classifer,cv=2,method='sigmoid')
#校准概率
classifer_sigmoid.fit(features,target)
#创建新的观察值
new_observation = [[2.6,2.6,2.6,0.4]]
#查看校准过的概率
classifer_sigmoid.predict_proba(new_observation)

—>

array([[0.31859969, 0.63663466, 0.04476565]])

• Reference: Python Machine Learning Manual