Deep Learning Model Training & Validation & Testing Process

For the training, verification and testing process of the model, there is a large room for manipulation. According to different testing methods, there are several different versions, which are recorded.

Mode 1: end of training & verification + test

Separate the training and testing of the model, save the model parameters in the training & verification phase, and load the model parameters in the testing phase to complete the test. The pseudo code is as follows:

Reference code: livebot/codes/transformer.py at master lancopku/livebot (github.com)

for i in epoch
    train()
    if batch_num satisfies XX:
         result = validate()
         if result satidfies threshold:
             save checkpoint()

# 加载模型参数进行测试测试
test()

This mode is widely used, and it is recommended to complete the test of the model when actually writing a program.

---

Mode 2: Validation driven testing during training

Including the testing process in the training phase, this method has relatively high coupling, but saves the step of model loading in the testing phase. The pseudo code is as follows:

Reference code: unknown (forgot the source...)

for i in epoch:
    train()
    result = validate()

    if result satidfies threshold:
        save checkpoint()
        test()

Note ⚠️: The result in the above code can be the calculation result of the loss of the current model on the validation set, or the calculation result of the evaluation index of the current model on the validation set.

The main difference between the two modes is whether the testing phase is embedded in the training & verification phase. Considering the cost of recurrence, the first mode is recommended to complete the training and testing of the model.

PS: The loss of the training set decreases while the loss of the test set keeps rising, indicating that the model is overfitting, which can be solved by increasing dropout or reducing the model size.