Epoch: Indicates the training process of going through all the samples in the training data set (and only once). In an epoch, the training algorithm will input all samples into the model for forward propagation, calculation of loss, back propagation and parameter update in the set order. An epoch usually consists of multiple steps.
batch: generally translated as "batch", which means a set of samples that are input into the model at one time. In the training process of the neural network, there are often a lot of training data, such as tens of thousands or even hundreds of thousands of data - if we put all these tens of thousands of data into the model at one time, the computer performance and neural network model learning The requirements for capacity, etc. are too high; then the training data can be divided into multiple batches, and then the samples of each batch are input into the model together in batches for forward propagation, loss calculation, back propagation and parameter update. But it should be noted that the word batch is generally not used much, and in most cases everyone only pays attention to the batch size.
Batch size: generally translated as "batch size", indicating the specific number of samples in a set of samples input to the model at one time during the training process . As mentioned earlier, in the process of neural network training, we often need to divide the training data into multiple batches; and the specific number of samples in each batch is specified by the batch size.
step: Generally translated as "step", it means that the model performs a parameter update operation in one epoch . In layman's terms, in the neural network training process, every time a batch of data is trained, a step is completed. In many cases, step and iteration mean the same thing.
iteration: Generally translated as "iteration", in most cases it means a step operation during the training process. An iteration includes the process of forward propagation, loss calculation, back propagation and parameter update in a step. Of course, in some cases, there may be subtle differences between step and iteration—sometimes iteration refers to the process of completing a forward propagation and back propagation, and step refers to an update of model parameters through an optimization algorithm. operate. But in most cases, we think that the two are the same.
When training the model, it is generally trained in batches. The total data set will be divided into n batches, and iter will be accumulated once for each batch of training.