Based on the simple identification codes Naive Bayes

Naive Bayes Theorem

Please refer to the principle:
http://www.ruanyifeng.com/blog/2011/08/bayesian_inference_part_one.html
https://www.cnblogs.com/TimVerion/p/11197043.html

I.e., the posterior probability = adjustment factor prior probability *

In the classification, the prior probability means the sample accounted for all categories category probabilities, the adjustment factor is the product of the probability of each sample characteristics, for example.

handsome or not	character	Not motivated	It is worthwhile to make friends
Shuai	it is good	Not motivated	not worth it
Not handsome	not good	Not motivated	not worth it
Shuai	it is good	Progress	worth it
Not handsome	it is good	Progress	worth it

Here is the prior probability means: worthy friends (1/2), does not deserve to make friends (1/2)

Adjustment factor is the feature you want to predict the probability of a sample, if a sample is not handsome | good | not motivated (example dispersion characteristics are not, because only two values, let this matter)

Well worth the post after posterior probability = 1/2 * 0 = adjustment factor, adjustment factor = not handsome account (1/2) in a worthy post data * Fortunately worth accounts (1) * does not deserve progress in accounting (0 )
not worth it to pay the posterior probability = 1/2 * 1/2 * 1/2 * 1/8 = 1

So this person is worthwhile to pay it, according to data 1/8> 0, it is not worth the pay. But because few samples, there is a probability of 0, which is a little problem, because practically impossible probability zero.

Smoothing parameters

So we need to introduce a smoothing parameter, to make this value is not 0

When we calculate the probability is not used directly: meet the requirements of sample / total samples, but samples meet the requirements of Alpha + / (total number of categories or sample feature tag + the number of categories * alpha), alpha and generally 1.0

I.e. prior probabilities: worth friends (2 + 1 / (2 * 4 + 1)) = 1/3, is not worth friends (2 + 1 / (2 * 4 + 1)) = 1/3

The calculation of the probability of the need for such computing features: other do not look, we do not look directly at it is worth making progress probability (that is, the probability is zero previously) = 0 + 1 / (2 + 1) = 1/4, attention here is the number of categories len (motivated, not motivated)
worth posterior probability post: 1/3 1/2 * 3/4 * 1/4 = 1/32
do not deserve the post-test probability: 1/3 * 1/2 * 1/2 * 3/4 = 1/16

Although probability is not worth the pay, but at least worth a post is not zero. If you do not know it, looking directly at the identification code, and then come back to see this.

sklearn in api

sklearn.naive_bayes.MultinomialNB(alpha=1.0, fit_prior=True, class_prior=None)
参数

alpha: smoothing parameters above said
fit_prior: whether to consider a priori probability, that is, the ratio of the share of each category
class_prior: given a priori probability array of
property
class_log_prior_: array, dimension (n_classes,), log probability for each category. n_classes is the number of categories, such as a ten categories each sample is 100, the array of (In 1/10, In 1/10, ...)
intercept_ : 同class_log_prior_
feature_log_prob_: array dimension (n_classes, n_features) a given class of feature experience log probability P (x_i | y). I did not understand
coef_ : 同feature_log_prob_
class_count_: number of sample arrays, the dimensions (n_classes,) for each category
classes_: array, dimension (n_classes,) for each category labels
feature_count_: number of samples per encountered (class, characteristic) array, during the fitting shapes (n_classes, n_features). I did not understand
the method
fit (x, y [, sample_weight]): Sample Use x and y labels for training
get_params (deep = True): Get all parameters of the model, deep do not know what
partial_fit (x, y [, classes, sample_weight]): a group of a number of training samples (when the sample is large enough)
predict (test_x): The sample test_x, return prediction y
predict_proba (test_x): Returns the sample test_x probability of belonging to each class, that is to say the return dimension (number of samples, k) of the two-dimensional array, all elements of each row and a 1-dimensional array, the array of length k .
score (test_x, y, sample_weight = None): The sample test_x prediction test_y, and then compare the actual fraction correct return y, sample_weight is a weight
set_params (** args): reset the model parameters

Of course, other naive Bayes classifier or regression, a difference is as follows:
wherein is a discrete variable, using a polynomial model (MultinomialNB)
when the characteristic is a continuous variable, the Gaussian model (GaussianNB)
Bernoulli model (BernoulliNB) and polynomial models are consistent, but the features are binarized (1,0)

Generation model

According to the above description that, as the simple codes, polynomial model may be used, Bernoulli model may also be used, because the picture has been binarized.

Known picture is 18x10 two-dimensional array, each element of the array is a number between 0 and 1. We feature 180 may be composed of, but are digital codes 0-9, the classification is calculated

Wherein assuming 180 are x1, x2, ..., x180, labeled 0-9, the number of samples of each label 120 is
a sample of probability of belonging to 0: P (0) = P0 ( x1) P0 (x2) .... P0 (x180 ) P total (0), P0 (x1) represents the proportion (probability) x1 share of the sample category 0, P total (0) 0 represents the proportion of the total sample of ( probability) that is 1/10, these values can all be obtained from training samples.

KNN code and essentially the same, as follows:

from sklearn import naive_bayes
import os
from PIL import Image
import numpy as np


def func():
    x = []
    y = []
    for label in os.listdir('train'):
        for file in os.listdir(f'train/{label}'):
            im = Image.open(f'train/{label}/{file}')
            pix = np.array(im)
            pix = (pix > 180) * 1
            pix = pix.ravel()
            x.append(list(pix))
            y.append(int(label))

    train_x = np.array(x)
    train_y = np.array(y)


    model = naive_bayes.MultinomialNB(alpha=1)

    model.fit(train_x, train_y)

    x = []
    y = []
    for label in os.listdir('test'):
        for file in os.listdir(f'test/{label}'):
            im = Image.open(f'test/{label}/{file}')
            pix = np.array(im)
            pix = (pix > 180) * 1
            pix = pix.ravel()
            x.append(list(pix))
            y.append(int(label))

    test_x = np.array(x)
    test_y = np.array(y)
    score = model.score(test_x, test_y)

    return score

if __name__ == "__main__":
    score = func()
    print(score)

On this simple identification codes, Naive Bayes can reach 100% correct. If the sample characteristics into 16, you will find naive Bayes KNN wrong place and are the same, are the same verification code identification became the same mistake.

Finally, I am learning some of the machine learning algorithms, for the record I need something I will share to blog and micro-channel public number, welcome attention. Usually, then the general share some content crawlers or Python. In addition, if the blog if there is an error, please also indicate.

This is the annotated data: https://www.lanzous.com/i8epywd

Finally, I am learning some of the machine learning algorithms, for the record I need something I will share to blog and micro-channel public number (python PATHS), welcome attention. Usually, then the general share some content crawlers or Python.