Deep Learning (17) - Metric Learning
Article directory
During the discussion yesterday, a big guy mentioned Metric Learning. After listening to him talk about the general idea, I found it a bit interesting, so I came down to learn about it, and simply recorded it.
1. What?
Metric learning is a method of spatial mapping. It can learn a feature space (Embedding), which will convert all data into feature vectors in a common space. Among these vectors, the more similar samples are, the closer the feature vector distance is. The distance between feature vectors of near and dissimilar samples is large to distinguish the data.
二、paired-based loss
1. Contrastive loss
Contrastive loss can make positive samples (more similar samples) as close as possible, and negative samples as far as possible, which can increase inter-class differences and reduce intra-class differences. But it needs to specify a fixed margin, because the margin is fixed, so there is a strong assumption implied here, that is, the sample distribution of each category is the same, but in general, this strong assumption may not be true.
For example, there is a data set with three animals, namely dogs, wolves, and cats. Intuitively, dogs and wolves are more similar, but the difference between dogs and cats is relatively large, so the margin between dogs and wolves should be smaller than the margin between dogs and cats. However, Contrastive loss uses a fixed margin. If the margin is set relatively large, the model may not be able to distinguish dogs and wolves well, and if the margin is set relatively small, it may not be able to distinguish dogs and cats well.
2. Triplet loss
The idea of Triplet Loss is to make the distance between negative sample pairs greater than the distance between positive sample pairs . During the training process, a pair of positive sample pairs and negative sample pairs are selected at the same time, and one of the positive and negative sample pairs is identical. Still taking the previous data of dogs, wolves, and cats as an example, first randomly select a sample, which is called an anchor sample, assuming that the sample category is a dog, and then select a sample of the same category as the anchor sample (another dog) , called positive, and let it form a positive sample pair (anchor-positive) with the anchor sample; then select a sample (cat) of a different category from the anchor, call it negative, and let it form a negative sample with the anchor sample Right (anchor-negative).
When the distance between the negative sample pairs is greater than the distance between the positive sample pairs by m, the loss is 0. It is considered that the current model has learned well, so the model is not updated.
3. Triplet center loss
The idea of Triplet Center loss is very simple. The original Triplet is to calculate the distance between the anchor and the positive and negative samples. Now the Triplet Center is to calculate the distance from the anchor to the center of the category where the positive and negative samples are located. The category center is the center of all sample embedding vectors of the category
4.N-pair loss
N-pair loss selects multiple negative sample pairs, that is, a pair of positive sample pairs, and selects all other samples of different categories as negative samples and combines them to obtain negative sample pairs. If there are N categories in the dataset, each positive sample pair Yii corresponds to N-1 negative sample pairs.
5. Quadruplet loss
Quadruplet loss consists of two parts:
- One part is the normal triplet loss, which allows the model to distinguish the relative distance between positive sample pairs and negative sample pairs.
- The other part is the relative distance between the positive sample pair and any other negative sample pair. This part of the constraint can be understood as the minimum inter-class distance is greater than the intra-class distance, regardless of whether these sample pairs have the same anchor
6. Lifted Structure Loss
The idea of Lifted Structure loss is that for a pair of positive samples, instead of distinguishing who is the anchor and who is positive in the sample pair, the distance between each sample in the positive sample pair and all other negative samples is equal to greater than a given threshold. This method can make full use of all samples in the mini-batch and mine all sample pairs.