Regarding the adjustment of one of the hyperparameters in the neural network training process ------ Batch Size. Let us discuss its impact on the performance of the model; from what aspects it has an impact on the performance of the model, and what kind of impact it has; whether these impacts can be eliminated or weakened by some methods.
- Definition of Batch Size
- Importance of Batch Size
- How to effectively select small batches and large batches during model training
- Reason analysis of poor performance in large batches, narrowing of performance gap
think
- Keeping the rest of the hyperparameters unchanged, only changing the Batchsize hyperparameter can see the different effects it has on the model. but! The performance of the model trained to the end is not only related to this parameter, but related to each hyperparameter and various conditions, which is the result of the combination of each hyperparameter. Therefore, only focusing on the optimal situation of a certain hyperparameter cannot better train a better model, but to grasp the combination of all hyperparameters as a whole to achieve the overall optimum.
- The combination of Batch Size and learning rate, the optimal match between the two can improve the performance of the model.
This article is:Understanding and thinking of hyperparameter Batch Size
For learning and sharing only