Before starting, a brief introduction to the cifar dataset:
[Introduction to cifar dataset]
Cifar-10 is a dataset for ubiquitous object recognition collected by two of Hinton's eldest disciples, Alex Krizhevsky and Ilya Sutskever. Cifar is an advanced science project institute led by the Canadian government. This project brought together many computer scientists, biologists, electrical engineers, neuroscientists, physicists, and psychologists to accelerate the process of DL. DL emphasizes adaptive perception and artificial intelligence, which is the intersection of computer and neuroscience. DM emphasizes high-speed, big data, statistical mathematical analysis, and the intersection of computer and mathematics.The CIFAR-10 dataset contains 60,000 32*32 color images, divided into 10 types, including 50,000 training images and 10,000 test images
The data of the dataset exists in an array of 10000*3072, the unit is uint8s, and 3072 stores a 32*32 color image. (3072=1024*3). The first 1024 bits are the r value, the middle 1024 bits are the g value, and the last 1024 bits are the b value. The biggest feature of this dataset is that the recognition is transferred to universal objects, and it is applied to multi-classification (sister dataset Cifar-100 reaches 100 categories, ILSVRC competition is 1000 categories).
1. Data set download
CIFAR-10 database download address: http://www.cs.toronto.edu/~kriz/cifar.html (download binary format)
2. Unzip to the relevant path
After the download is complete, unzip it to .../caffe-master/data/cidar10/, and put it together with get_cifar10.sh. In fact, the function of the ./get_cifar10.sh command is "download and unzip", only But it's in a linux environment. Among them, data_batch_1.bin, data_batch_2.bin, data_batch_3.bin, data_batch_4.bin and data_batch_5.bin contain 50,000 color images in 10 categories (5000 per category) in the database, which are used for training, while test_batch_. The bin contains 10,000 images in 10 categories (1000 images in each category) for testing.
3. Convert the database to lmdb format
Enter the path where convert_cifar_data.exe is located.../caffe-master/Build/x64/Release/, execute in cmd:
After the command is executed, you can see the two generated folders cifar10_test_lmdb and cifar10_train_lmdb under the .../caffe-master/examples/cifar10/ path, which are the converted databases corresponding to the test database and the training database respectively. Next, average the training images in db format, enter the path where convert_cifar_data.exe is located.../caffe-master/Build/x64/Release/, and execute in cmd:
After the command is executed, you can see the generated mean.binaryproto under the .../caffe-master/examples/cifar10/ path.
From this we get:
1--cifar10_train_lmdb--------training sample dataset
2--cifar10_test_lmdb---------test sample dataset
3--mean.binaryproto------- data set mean file (for mean subtraction)
4. Training
The CNN consists of convolutional layers, POOLing layers, nonlinear transformation layers, and a local contrast-normalized linear classifier on top. There are a bunch of *.prototxt under the .../caffe-master/examples/cifar10/ path, which are all used to modify configuration parameters. Since the platform we use does not have NVIDIA GPU, we can only use CPU for training, so we changed solver_mode:GPU in cifar10_quick_solver.prototxt and cifar10_quick_solver_lr1.prototxt to solver_mode:CPU. These two parameter profiles are used to coordinate the optimization of the model, such as the learning rate.
When training or testing, use caffe.exe in Release mode, enter the .../caffe-master/Build/x64/Release/ path, use the dos command line, that is, execute cmd, and execute the following commands:
After the above command is executed, two files, cifar10_quick_iter_4000.caffemodel.h5 and cifar10_quick_iter_4000.solverstate.h5, will be generated, of which cifar10_quick_iter_4000.solverstate.h5 will be used in further training, and the cifar10_quick_iter_4000.caffemodel.h5 model weight file can be used for data The test of the set (it is not necessary here, because there is a deeper training below, which will generate a deeper model weight file cifar10_quick_iter_5000.caffemodel.h5). The process lasts about half an hour.
On the basis of the previous step, further execute the following commands:
After the above command is executed, two files, cifar10_quick_iter_5000.caffemodel.h5 and cifar10_quick_iter_5000.solverstate.h5, will be generated. In this example, the cifar10_quick_iter_5000.caffemodel.h5 model weight file is used for prediction.
It should be noted that when the above command is executed, an error will be reported, indicating that the file cannot be found or cannot be written:
This is a problem caused by the file path. In order to solve this problem, I have changed the path in each file. You can modify the corresponding file path according to the error content. The path in the file content to be modified is generally cifar10_quick_solver.prototxt and The path corresponding to net and snapshot_prefix in cifar10_quick_solver_lr1.prototxt, and the path corresponding to mean_file and source in cifar10_quick_train_test.prototxt. Here, I changed the paths in cifar10_quick_solver_lr1.prototxt and cifar10_quick_train_test.prototxt to absolute paths.
In the original prototxt file, all the paths related to the path are written very vaguely, and the upper-level path is omitted. When you move the file a little, and run according to the example, there will be path inconsistencies. Therefore, it is recommended to change all paths to absolute paths first.
The following will briefly describe or explain the contents of the configuration file:
net: network description file for training and prediction
test_iter: number of iterations in the prediction phase
test_interval: how many times each iteration during training, make a prediction
base_lr, momentum, weight_delay: network The basic learning rate, impulse and weight decay
lr_policy: The decay policy of the learning rate
display: After how many iterations, print a running log on the screen
max_iter: The maximum number of iterations
snapshot: How many iterations print a snapshot
solver_mode: caffe's Solving mode, select GPU or CPU according to actual situation
5. Test
In fact, the prediction is to use the trained model for prediction. For this example, the cifar10_quick_iter_5000.caffemodel.h5 model weight file is used for prediction.
Go to the .../caffe-master/Build/x64/Release/ path and execute the following command in cmd:
Among them:
test : Indicates only prediction (forward propagation calculation), no parameter update (backward propagation calculation) -model ../../../examples/cifar10/cifar10_quick_train_test.prototxt :
Specifies the model description text file-
weights ../../../examples/cifar10/cifar10_quick_iter_5000.caffemodel.h5 : Specify the model weight file, that is, the pre-trained model or weight file
-iterations 100: Specifies the number of iterations, that is, the number of samples participating in the test.
When using the cifar10_quick_iter_5000.caffemodel.h5 model weight file for prediction, the final result is as follows, and the accuracy rate is about 0.75:
If the cifar10_quick_iter_4000.caffemodel.h5 model weight file is used for prediction, the accuracy rate is 0.7159:
It shows that the number of iterations still has a certain influence on the final result.