Recently, I have been watching the application of semi-supervision in hyperspectral image classification, and I plan to write a series to summarize and sort out some classic papers with good results as a supervision.
1. Semi-supervised can be roughly divided into five methods
① Generative model
Disadvantage: Generative models are built based on strict assumptions, e.g. training samples should follow a Gaussian or other distribution
②Self-training
The practice of Self-Training is as follows:
- Train a classification model M with labeled dataset A
- Use the model to make predictions on the unlabeled dataset B
- Add K samples with high confidence in the prediction results, together with their Labels, into training data A, and delete them from B
- Go back to step 1.
Disadvantage: If a misclassified sample is added to the original training set, then in the subsequent training process, its mistakes will only get deeper and deeper, and it will induce other samples to make mistakes, which is also self-training The biggest flaw of the model.
③ The process of Co-training
collaborative training is as follows:
Assume that the data has two feature representations, such as image features (X1, Y1) and text features (X2, Y2). There are also two features for unlabeled data. The algorithm is as follows:
- Two classification models F1, F2 are obtained by training from (X1, Y1), (X2, Y2) respectively
- Use F1 and F2 to make predictions on unlabeled data, respectively
- Add the top K samples with the highest confidence predicted by F1 to the training data set of F2
- Add the top K samples with the highest confidence predicted by F2 to the training data set of F1
- go back to step 1
shortcoming:
④Transductive Support Vector Machine (TSVM)
(SVM may lead to a local minimum because of the non-convex loss function)
The more famous semi-supervised support vector machine is TSVM, which is a learning method for binary classification problems. TSVM attempts to consider possible label assignments to unlabeled samples, i.e., try to treat each unlabeled sample as a positive or negative example, and then, among all these results, seek a Separation hyperplane that maximizes the separation on unlabeled samples with label assignments). Once the hyperplane is determined, the final label assignment for unlabeled samples is its prediction.
Disadvantage: easy to fall into local optimal solution
⑤Graph-based methodDisadvantages: computationally expensive and unable to generate labels for a given sample
二. Semi-supervised classification for hyperspectral imagery based on spatial-spectral Label Propagation（SS-LPSVM）
Because the graph cannot fully represent the inherent spatial distribution of the data, the paper adds spatial information and spectral information for semi-supervised classification and class label propagation. The main idea: first select a part of the labeled samples, use the class transfer to find a part of the unlabeled samples (the neighborhood of the labeled samples) from the unlabeled samples, and then get their categories according to the transfer. Finally, the newly added labeled samples and the initial labeled samples are put together to train the SVM for final classification.
The main idea of graph-based semi-supervised classification algorithms is to assign class labels to unlabeled samples by building a graph to minimize a defined energy function. For the energy function, two conditions should be satisfied:
(1) the loss function is the smallest, that is, the predicted labels of the labeled samples should be as similar as possible to the existing labels;
(2) the smoothing function is the smallest, which means that the two adjacent samples have the most may belong to the same class.
Let the vector of labels be