1. Summary
Success is partly due to the depth of the neural network of well-marked large-scale training data. However, with the growing size of modern data collection, labeling access to information extremely difficult. This paper proposes an incremental depth study based on semi-supervised self-labeling method generates network-style confrontation, the method to facilitate the training process through virtual tag data distribution continue to be unmarked. Specifically, the virtual tag allocation process, the paper introduces a method based on self-marking time. Then, in order to allocate more virtual tags dynamically to the data in the training process, the paper adopts incremental update label screening and phased approach. Finally, the paper introduces a further factor balancing item (balance factor Term, BT), loss of information during the balance training samples.
2. Introduction
For semi-supervised classification against network using formula (Gans), most networks are used to produce k outputs by modifying a conventional discriminator corresponding to the GAN k classes. To further utilize unlabeled training data, typically generate an additional section (k + 1) th class from the generator to enhance the ability to identify discriminator. The latter features can extract more information for distinguishing the true data and false data.
The paper dedicated to exploring from an incremental marking method (the ISL -GaN), and embedded into robust SSL (SSL) framework, in order to improve the classification performance in the field of GAN.
3. Methods
First, the accuracy of prediction of the top labels, most of the training data, including tag data and unlabeled data, in the training process have been correct prediction. To further test the robustness of the model of the noise marker, the paper added some sample error flag in the model training, found that a certain percentage of label error does affect the accuracy of the final test.
The following describes the paper model proposed. As shown in FIG. 1, the proposed model consists of two parts: the first part is based on the consistency of the semi-supervised GAN model. The second part is responsible for allocating virtual label unlabeled data, the epoch at regular intervals, assign a virtual tag for high reliability data to update the tagged training data set.
Known for the different stages of the training sample output of the network stable, low probability of misclassification, easily misclassified samples often occurs near the edge of classification, which will inevitably lead to instability of the sample output. With this in mind, each training sample in order to maintain a relatively stable and secure virtual label, select the paper calculated the average number of historical output to ensure stability.
Taking into account the final study sample of high data rate label correctly predicted by the semi-supervised way, the paper using this method and by setting a virtual label unlabeled data to update the training data set. If a non-labeled sample was assigned several times with a category label, then put this class label as a virtual label of the sample. During training, one unlabeled samples assign a virtual tag, can effectively increase the number of SSL labeled sample, thereby increasing the classification accuracy.
Here, we need to set a threshold of credibility as a sample virtual tag is assigned. This threshold can not be too low, not too high. If set too high, say 100% confidence, will result in: When we use this part of the data model is updated when the calculated loss value is 0, no further updates to the model. Too low, resulting in this part of the data can not be trusted, and we use it for this idea is contrary to the training data. Therefore, in order to increase the contribution of virtual labeled samples have been fitted low-loss model, the paper will balance factor item (BT) exp (pin) into the original cross-entropy loss CE, the final loss of oversight by the following formula representation (detailed see original paper model loss function, here only the balance factor given term introduced by reference formula):
PI indicates the probability of the sample belonging to class i, yi representative of one-hot encoded value of the class i. The parameter n is the control over the weight loss of weight balance factor, the default is 2.0.
4 Conclusion
Experimental results show that the method can be obtained MNIST, the results of the latest SSL and SVHN CIFAR-10 data sets. In particular, the model of the paper in the sample labeling conditions less favorable performance. For 1000 only labeled image data sets CIFAR-10 can be achieved 11.2% of measurement error, and 1000 for 500 SVHN labeled data set, can achieve almost the same performance test error of 3.5%.
