Batch processing (Batch Processing) refers to the way of processing multiple samples at the same time when updating model parameters in each iteration in deep learning .
When batch processing, it should be noted that the number of elements in the corresponding dimension must be consistent
Regarding the previous example of handwritten digit recognition:
represented by a graph, it can be found that the number of elements in the corresponding dimensions of the multidimensional array is indeed the same. In addition, it can also be confirmed that the final result is a one-dimensional array with 10 output elements.
Now consider the case of packing multiple input images.
We want to use the predict function to pack and process 100 images at a time. To do this, you can change the shape of to 100 x 784 and pack 100 images as input data. words represented by diagrams
It can be found that the shape of the final output data is 100 x 10.
This packaged input data is called batch .
Batch processing is of great benefit to the operation of the computer, which can greatly shorten the processing time of each image .
Below we implement batch-based code implementation.
x,t = get_data()
network= init_network()
batch_size = 100 # 批数量
accuracy_cnt = 0
for i in range(0,len(x),batch_size):
x_batch = x[i:i+batch_size]
y_batch = predict(network,x_batch)
p= np.argmax(y_batch,axis=1)
accuracy_cnt += np.sum(p == t[i:i+batch_size])
print("Accuracy:"+str(float(accuracy_cnt) / len(x)))
By extracting the batch datax[i:i+batch_size] from the input data , and then, by obtaining the index of the element with the largest value , it should be noted that axis =1 is given here, which also means that in the 100 x 10 array, along the first dimension ( Take the 1st dimension as the axis) to find the index of the element with the largest value ( the 0th dimension of the matrix is the column direction, and the 1st dimension is the row direction ), see the following example:argmax()
>>>x = np.array([[0.1,0.8,0.1], [0.3，0.1，0.6], [0.2，0.5，0.3], [0.8，0.1，0.1]])
>>>y=np.argmax(x, axis=1)
>>>print(y)
[1 2 1 0]
When implementing batch processing, you need to pay attention to the following aspects:
Memory limit : The batch size should be set according to the memory size of the hardware device to prevent memory overflow.
Data division : When dividing the training data into multiple batches, it should be ensured that the sample distribution of each batch is similar to avoid training deviation.
Learning rate adjustment : The use of batch processing may have an impact on the choice of learning rate, and the learning rate needs to be adjusted accordingly.