"Sklearn source machine learning notes" series of tutorials to Scikit-learn 0.19.x library, based on the series from the start Sklearn package source notes were understood algorithms, in-depth study for some demo sklearn document.
Due to limited capacity, lack of support please correct me a lot, you have any ideas are welcome leave comments!
More about my study notes, welcome your interest in " Wuhan AI algorithm study " Public number, Sklearn official website latest original English document download , public No. reply " sklearn ."
This article is divided into three parts " [Gauss naive Bayes] ", " [polynomial naive Bayes] ", " [Bernoulli Naive Bayes] " to expand the total reading time of about 10 minutes.
About Naive Bayesian method principle article, article number visible public, "Li Hang" statistical learning methods, "the study notes - Chapter 4: Naive Bayes Law"
Naive Bayes algorithm, commonly divided into naive Bayes Gaussian, polynomial Bernoulli Naive Bayes and three kinds of naive Bayes, Sklearn is encapsulated for conventional three kinds of naive Bayes method sklearn .naive_bayes module to encapsulate naive Bayes usual method.
Naive Bayes [Gauss]
Naive Gaussian Bayesian algorithm is a special type of NB algorithm, which is used especially when the feature value having continuous time, while assuming that all features Gaussian distribution , such as by raising the predicted sex, and weight, since the increase in body weight and wherein Numerical values are continuous. Sklearn in GaussianNB () class is a concrete implementation naive Bayesian classification method.
navie_bayes.GaussianNB(priors = None, var_smoothing)More parameters priors priori probabilities, general default to None, when the default is None, the prior probability is calculated automatically according to the number of classes and number of samples, when given a priori probability to give the subject. var_smoothing to solve the problem of discrepancies between various dimensions, in order to calculate error-free.
#np.var(X, axis=0).max()得到个特征均值最大值
#如果维度之间的数据差异比率太小，则会导致数值错误。
#为了解决这个问题，我们通过epsilon人为地增加方差，这是最大维度标准差的一小部分。
elf.epsilon_ = self.var_smoothing * np.var(X, axis=0).max()
self.sigma_[:, :] += self.epsilon_
self.sigma_[:, :] -= self.epsilon_
GaussianNB () class attributes comprising:
- class_prior_: probability of each sample;
- class_count_: the number of samples in each category;
- theta_: each category mean value for each feature;
- sigma_: each category variance of each feature;
- epsilon_: absolute value variance (adjusted variance);
import numpy as np
X = np.array([[-1, -1], [-2, -1], [-3, -2], [1, 1], [2, 1], [3, 2]])
Y = np.array([1, 1, 1, 2, 2, 2])
from sklearn.naive_bayes import GaussianNB
clf = GaussianNB(priors=None, var_smoothing=1e-09)
clf.fit(X, Y)
print "==Predict result by predict==直接给出测试集的预测类别输出"
print(clf.predict([[-0.8, -1]]))
print "==Predict result by predict_proba==给出测试集样本在各个类别上预测的概率"
print(clf.predict_proba([[-0.8, -1]]))
print "==Predict result by predict_log_proba==给出测试集样本在各个类别上预测的概率的一个对数转化"
print(clf.predict_log_proba([[-0.8, -1]]))
Naive Bayes [polynomial]
And more for discrete classification features , such as text classification word count, the number to appear as a characteristic value.
sklearn.naive_bayes.MultinomialNB(alpha=1.0, fit_prior=True, class_prior=None)MultinomialNB () class parameter:
- alpha: smoothing factor a priori, by default equal to 1, indicates the Laplace smoothing when equal to 1, if not smoothed, when the value of one dimension not outdated features appear in the training sample, will cause the conditional probability value is 0, so leading to the posterior probability is zero.
- fit_prior: whether to go to class to learn a priori probability of default is True
- class_prior: prior probability of each category, if not specified, the model will learn the data, the same a priori probability of each category, the total number is equal to one class marker-N.
MultinomialNB () class attributes:
- class_log_prior_: prior probability of each class after smoothing
- intercept_: naive Bayes linear model corresponding to the same value and class_log_prior_
- feature_log_prob_: given feature category number probability (conditional probability) pairs. = Conditional probability of the feature (the number of occurrence of the specified feature at the specified class Alpha +) (wherein all occurrences and the number of possible values + classes specified class * alpha) /
- coef_: naive Bayes linear model corresponding to the same value and feature_log_prob
- class_count_: the number of samples in the training sample corresponding to each category
- feature_count_: number of occurrence of each feature in each category
import numpy as np
X = np.random.randint(5, size=(6, 100))
y = np.array([1, 2, 3, 4, 5, 6])
from sklearn.naive_bayes import MultinomialNB
clf = MultinomialNB()
clf.fit(X, y)
MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)
print(clf.predict(X[2:3]))
Bernoulli Naive Bayes []
As with the polynomial model, Bernoulli Naive Bayes applicable to discrete features situation , but Bernoulli model values for each feature must be 0 or 1 .
sklearn.naive_bayes.BernoulliNB(alpha=1.0, binarize= 0, fit_prior=True, class_prior=None)BernoulliNB () class parameter:
- alpha: smoothing factor, in accordance with polynomials alpha.
- binarize: Sample Characteristics binarization threshold value, 0 is the default. If no, the model considers all features have already been binarized form; if a specific input value, the model is greater than this value will partially classified as a class, it is classified as less than another.
- fit_prior: whether to go to class to learn a priori probability of default is True
- class_prior: prior probability of each category, if not specified, the model will learn the data, the same a priori probability of each category, the total number is equal to one class marker-N.
BernoulliNB () class attributes:
- class_log_prior_: priori after smoothing the log probability of each category.
- feature_log_prob_: to log probability of a given feature class experience.
- class_count_: the number of each sample during fitting.
- feature_count_: feature quantity of each of the fitting process.
import numpy as np
X = np.random.randint(2, size=(6, 100))
Y = np.array([1, 2, 3, 4, 4, 5])
from sklearn.naive_bayes import BernoulliNB
clf = BernoulliNB()
clf.fit(X, Y)
BernoulliNB(alpha=1.0, binarize=0.0, class_prior=None, fit_prior=True)
print(clf.predict(X[2:3]))
[references]
[1] Naive Bayesian Inference and three common models
[2] Sklearn Parameter Description - Bayesian
[. 3] scikit Learn naive Bayes-use library Summary
