Why use transfer learning: There are too many model parameters and the amount of data is too small. Direct training can easily lead to overfitting.
How to use transfer learning: 1: Train on a large dataset and finetune on a small dataset. 2: Choose whether to freeze the layer according to the situation.
Background: Training on CItyscape, cityscape has 2975 pictures for training, and then in order to get a result on KITTI semantic segmentation: I have
read other papers and there are two options, 1: direct training using ten-fold cross-validation. 2: Fine-tune KITTI using weights trained on the Cityscape dataset.
For option 1: direct training, the result is still overfitting, the test set in KITTI can reach 70 mIoU, but submitted to the website to test the unseen image, the result is only 45. For option 2: fine-tuning is also divided into frozen encoder
parameters Fine-tune the decoder, and load pre-trained weights as initialization parameters, and retrain.
First of all the first one:
checkpoint = torch.load(args.finetune)
model.load_state_dict(checkpoint['state_dict'])
print(f'Loaded weights for finetuning: {
args.finetune}')
for name, param in model.named_parameters():
f 'encoder_rgb' in name or 'encoder_depth' in name :
param.requires_grad = False
The second method: direct loading Comment out the param.requires_grad = False above, that is, retrain. The fine-tuning is explained in the last five minutes.
It should be noted that the preprocessing of the Cityscape dataset should be unified with that of KITTI . If it is not unified, the convergence speed will be very slow and the result will be relatively poor.