1 Introduction
In supervised learning areas, mankind has made great progress, but it also means that we need a lot of data tagged to train the model, these algorithms need to scan the data again and again to find the optimal model parameters. However, the reality production activities, the relative lack of labeled data, vast amounts of unlabeled data are not fully utilized, this post will introduce plain under a semi-supervised method - of counterfeit labels.
2. What is a pseudo-label
Pseudo-label is to add predictions reliable test data to training data. The process of establishing the dummy label about five steps: (1) model using training data; tag (2) predict unknown test data set; (3) adding the prediction value reliable test data in the training data; (4) the use of a combination of the new data model train or fine tune the first step of the model; (5) use the new model to predict the test data set.
3. The training process
This post is a reference TGS Salt Identification Challenge champion on the program. This scheme for the lack of seismic markers in the image data, the proposed semi-supervised method using a large amount of unlabeled data. Self-image using unlabeled training, the proposed self-training process is an iterative process, to extend the data set by alternately tagged training model and pseudo label, the model K wheel train. FIG detailed procedure is as follows:
from one iteration procedure K training process, there are two round steps: (a) using the pseudo extensible markup tag training model data set; (b) updating the dummy tag untagged data.
In the first round, we use only the GT (Ground Truth) tags to train the model. Then, we predicted pseudo tags all untagged data by assigning the most probable class for the test set for each pixel in the image. We can not reliably predict with low confidence of counterfeit labels by deleting filtered out.
The next round, we first joint use of counterfeit labels and tags GT model retraining; then use the new model to update all the pseudo-label unlabeled data. Before each round of self-training model of the right to reset the weight is crucial, and not in the course of several rounds of training in the accumulation of errors in the pseudo-label.
To further improve the robustness of the pseudo-generated label, to prevent overfitting single error model, we trained a joint CNNs having different backbone structures set. In this case, the dummy tag is obtained by integration of all the prediction model generated by averaging, integration of each model using the prior knowledge of a reliable integrated and expressed in the polymerization and the dummy tab.
4. References
Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks[J];
https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/discussion/107981;
