Machine learning 0007 How much data should be trained each time batch_size should be set
Here are some of the experiences collated:
1. The larger the value of batch_size, the faster the training (convergence) speed; the smaller the value of batch_size, the slower the training (convergence) speed.
2. The smaller the value of batch_size, the better the model generalization ability; the larger the value of batch_size, the worse the model generalization ability.
3. When batch_size=1, the model is difficult to converge and may not converge. Convergence is only possible in very special cases.
In general, batch_size should be set to 2^n. Some people on the Internet say that it is convenient for memory/video memory allocation, and the GPU will be faster under the condition of 2^n. The batch cannot be too large or too small.
To sum up: The recommended values for batch_size are 32, 64, 128