table of Contents
Experiment 2 uses Bayesian decision theory to realize handwritten digit recognition
1. The purpose of the experiment
1. The principle of the classifier
2. Realize the principle of handwritten digit recognition algorithm
Fourth, the experiment summary
2. Analyze the influence of different templates on the experimental results.
4. Analyze the accuracy of each digit recognition under the same threshold under the same template.
5. Analyze the P matrix in the program.
Experiment 2 uses Bayesian decision theory to realize handwritten digit recognition
1. The purpose of the experiment
Familiar with and master the Bayesian decision theory, and use the relevant knowledge of Bayesian decision theory to realize the recognition algorithm of handwritten digits, and analyze the influence of the main parameter changes on the recognition results.
2. Experimental principle
1. The principle of the classifier
Bayesian classification is a general term for a class of classification algorithms. This type of algorithm is based on Bayes' theorem, so it is collectively referred to as Bayesian classification algorithm. It is a class of algorithms that use probability and statistics for classification. The Bayesian classifier is an optimal classifier, and the error generated by the results obtained by this classification method is always the lowest. The basis of this method is the Bayesian formula, which can calculate the probability that the sample belongs to each category, also known as the posterior probability.
The main idea is: for a given item to be classified, the probability of each category appearing under the condition of the occurrence of this item is solved, and the class corresponding to the maximum value obtained is considered to belong to this category.
2. Realize the principle of handwritten digit recognition algorithm
For a given sample set, the training library has 60,000 images of handwritten digits, and the test library has 10,000 images of handwritten digits. Based on the above-mentioned classifier, a handwritten digit recognition classifier is designed based on Bayesian decision theory.
Figure 2.2.1 A sample of handwritten digits "0" Figure 2.2.1 A sample of handwritten digits "8"
3. Experimental results
To the base to select a different threshold, and considering the image templates and algorithms actual efficiency on the choice of 4 * 4 template , to take the threshold value (here the threshold refers to the template pixel value is not the total number of 0 elements) of 4 which The final average correct rate was 75.6% . Under different thresholds, the correct rates of various samples are shown in Table 3-3.
Table 3.1 Comparison of the results of handwritten digit recognition under different thresholds under the 4*4 template
| Threshold category |
3 |
4 |
5 |
| 0 |
86.9% |
86.2% |
84.0% |
| 1 |
94.9% |
94.6% |
94.4% |
| 2 |
75.0% |
71.8% |
68.2% |
| 3 |
74.0% |
76.9% |
76.6% |
| 4 |
71.0% |
73.0% |
72.3% |
| 5 |
53.0% |
55.7% |
55.0% |
| 6 |
83.1% |
81.4% |
79.1% |
| 7 |
78.3% |
79.0% |
80.4% |
| 8 |
62.1% |
62.4% |
60.8% |
| 9 |
74.4% |
75.6% |
73.0% |
| average |
74.4% |
75.6% |
73.0% |
Fourth, the experiment summary
1. Experimental process
First, the file data downloaded minst.mat binarization processing is performed, a non-0 element is set to 1, the remaining pixels to 0. The flow of FIG. 4.1 , a digital handwriting recognition algorithm steps are as follows:
- According to the training samples, the prior probability is approximately calculated from the number of samples and the total number of samples:
i=0,1,2,...,9, where Ni is the number of samples with number i, and N is the total number of samples.
- According to the training sample, calculate Pj(Wi), and then calculate the class conditional probability P(X|Wi):
j=0,1,2,...,49, that is, the probability estimate that the jth component of X is 1 under the condition that X belongs to the Wi class.
- For each sample X in the test sample set, the conditional probability is calculated as:
- Use Bayesian formula to find the posterior probability:
- The category of the maximum value of the posterior probability (0-9) is the category of the handwritten X of the test data.
Figure 4.1 Handwritten digit recognition flow chart
2. Analyze the influence of different templates on the experimental results.
The initial size of the downloaded single sample image is 28*28. We can use the templates of 2*2, 4*4, and 7*7 to conduct experiments. The experiment found that when the 2*2 template is selected, the recognition effect is good, but the correct rate is OK. It reaches 82% , but the running time is too long. If 4*4 template is selected, the correct rate is 75.6% at the appropriate threshold . When the 7*7 template is selected, the correct rate is less than 50% , so it is not considered.
3. Analyze the influence of different thresholds on the experimental results under the same template.
Under the premise of selecting 4*4 templates, by selecting different thresholds, as shown in Figure 4.3.1, the picture after the number "0" is displayed after dimensionality reduction, when the Threshold is 3, 4, 5, and 6, as shown in Figure 4.3. As shown in 1 , we found through comparison that when the threshold value is 4, the correct rate achieves the maximum value of 75.6% .
Figure 4.3.1(1) The threshold is 4 Figure 4.3.1(2) The threshold is 5
Figure 4.3.1 The effect of number "0" after dimensionality reduction
Figure 4.3.2 The influence of different thresholds on the experimental results
4. Analyze the accuracy of each digit recognition under the same threshold under the same template.
On the premise that the 4*4 template is selected and the threshold is 4, as shown in Figure 4.4 below, we can find that the recognition accuracy of different numbers is different. The number "1" was recognized with the highest correct rate of 94.6% , and the number "5" was recognized with the lowest correct rate of only 55.7% .
Figure 4.4 The correct rate of different numbers being recognized
5. Analyze the P matrix in the program.
The P matrix is obtained according to the feature vector of the training set. Under the 4*4 template, the single image of the sample is reduced to 7*7, and the size of the finally obtained P matrix is 10*49. The P matrix is recorded This sample has a probability of 1 corresponding to the position of 7*7 pixels. For example, P(i,j) represents the probability that the j-th pixel is 1 for the i-type sample. The approximate elements of the P matrix are shown in Table 4.5 below:
Table 4.5 Partial display of P matrix
| i j |
1 |
10 |
20 |
30 |
40 |
| 0 |
1.69E-04 |
8.29E-01 |
2.23E-02 |
7.41E-01 |
9.55E-02 |
| 1 |
1.48E-04 |
5.09E-01 |
1.48E-04 |
2.75E-01 |
4.45E-03 |
| 2 |
1.68E-04 |
9.02E-01 |
1.68E-04 |
8.77E-01 |
3.44E-01 |
| 3 |
1.63E-04 |
9.54E-01 |
1.63E-04 |
1.81E-01 |
1.14E-01 |
| 4 |
1.71E-04 |
2.11E-01 |
3.76E-03 |
6.28E-01 |
7.15E-02 |
| 5 |
1.84E-04 |
6.43E-01 |
5.77E-02 |
2.97E-01 |
1.08E-01 |
| 6 |
1.69E-04 |
7.18E-01 |
1.35E-03 |
9.74E-01 |
3.58E-02 |
| 7 |
1.60E-04 |
3.74E-01 |
2.23E-03 |
1.01E-01 |
1.10E-02 |
| 8 |
1.71E-04 |
8.75E-01 |
3.04E-02 |
7.95E-01 |
8.10E-02 |
| 9 |
1.68E-04 |
6.51E-01 |
1.18E-03 |
4.18E-01 |
6.02E-02 |
Through the P matrix, we can see that in different columns, that is, at different pixel positions, the probability value of the same type of sample being 1 is different. For vertical comparison, for the same position of the pixel, the probability of different types of samples is also different. Yes , this shows that the value of the feature vector corresponding to different samples is different, which is the key to our realization of this program.
6. Experimental experience
This experiment took a long time. For the programming of the experimental code, I felt that I had basically mastered the theoretical knowledge of Bayesian. After discussing with my classmates, I decided to program independently without looking at the code provided by the teacher. However, various problems appeared in the process of programming. First of all, the size of the data set was not clearly understood and handled blindly. Finally, the code that I worked hard to write had a low accuracy rate. Later, after consulting others, I learned that the downloaded data set is stored in accordance with one sample per line. In addition, when writing dimensionality reduction and calculating probability, I have never understood some python matrix operation functions before, and nested some for loop statements, resulting in very low efficiency of my program. This also shows that I need to improve in this area. At the same time, under the guidance of the teacher, I understand how to analyze the influence of each parameter on the experimental results, and pay attention to the standardization of writing. In the end, I successfully completed this experiment. Although it took a long time, there is always a reward for the effort. While gaining knowledge, I have further honed my will when facing difficult problems.
Reference code data set
Download link: https://download.csdn.net/download/weixin_43442778/16016855
recommended article
- 700 sets of personal resume templates (postgraduate entrance examination and postgraduate research work): https://mp.csdn.net/editor/html/114280230
- Artificial intelligence review knowledge points at the end of the 2019 fall semester: https://mp.csdn.net/editor/html/104431474
- The design and implementation of Fisher linear classifier, the design and implementation of perceptron algorithm: https://download.csdn.net/download/weixin_43442778/16017212
Welcome everyone to pay attention to the [Xiaoguoguo Senior] WeChat public account for more information and source code, and look forward to your praise and support!