Classification using Naive Bayes
First, the naive Bayes formula is as follows
2. The formal definition of Naive Bayesian classification is as follows
1. Let x={a 1 , a 2 ,..., am } be an item to be classified, and each a is a feature attribute of x.
2. The existing classified category set C={y 1 , y 2 , .., y 3 }.
3. Calculate P(y 1 |x), P(y 2 |x), ..., P(y n |x). ( The third step of the calculation is the key, see the fourth point for the detailed algorithm)
4. If P(y k |x)=max{P(y 1 |x), P(y 2 |x), ..., P(y n |x)}, then x∈y k .
2. Practical application example of Naive Bayes classification
If there are two types of fruits, their names and properties are as follows:
Apples (color, origin, price, size, peak season)
Watermelon (color, origin, price, size, peak season)
The following steps correspond to the steps of the second major point, which can be combined to see
1. Item to be classified x={red, Shandong, 3, small, December}
2. The existing classified category set C={apple, watermelon}
3. Calculate P(apple|{red, Shandong, 3, small, December}) and P(watermelon|{red, Shandong, 3, small, December}).
P(apple|{red, Shandong, 3, small, December}) means:
There is a kind of fruit, it is red, the origin is in Shandong, the retail price is 3 yuan, the shape is small, and the peak sales season is in December.
So what is the probability that it is a watermelon.
4. If P(apple|{red, Shandong, 3, small, December}) = max{P(apple|{red, Shandong, 3, small, December}), P(watermelon|{red, Shandong, 3, small, December }) },
Then {red, Shandong, 3, small, December} is an apple
4. Calculate P(y 1 |x), P(y 2 |x) , ...,P (y n | x)
Since the denominator is constant for all categories, it is only necessary to maximize the numerator:
Simplified to the following formula:
5. The process of using Naive Bayes classification
1. Training data to generate a training sample set: TF-IDF.
2. Calculate P(y i ) for each class.
3. Calculate the conditional probability of all partitions for each feature attribute.
4. Calculate P(x|y i )p(y i ) separately for each category .
5. Take the largest item of P(x|y i )p(y i ) as the category of x.