Deep learning paper: Student-Teacher Feature Pyramid Matching for Anomaly Detection and its PyTorch implementation
Student-Teacher Feature Pyramid Matching for Anomaly Detection
PDF: https://arxiv.org/pdf/2103.04257v2.pdf
PyTorch code: https://github .com/shanglianlm0525/CvPytorch
PyTorch code: https://github.com/shanglianlm0525/PyTorch-Networks
1 Overview
STFPM follows the strengths of the student-teacher framework but greatly expands on accuracy and efficiency. Specifically, STFPM uses a strong network pre-trained on image classification as a teacher to incorporate its knowledge into a single student network with the same architecture. In this case, the student network learns the distribution of anomaly-free images by matching the features of the anomaly-free images with the corresponding features of the pre-trained network. The transfer of this step preserves the key information as much as possible. In addition, to enhance scale robustness, STFPM embeds multi-scale feature matching into the network. This hierarchical feature matching strategy enables the student network to receive multi-level knowledge mixtures from the feature pyramid under stronger supervision and is able to detect Anomalies of all sizes. The feature pyramids of the teacher network and the student network are compared for prediction, with larger differences indicating a higher probability of anomalies occurring.
advantage:
- First, in the one-step distillation process, useful knowledge can be well transferred from the pre-trained network to the student network with the same structure.
- Secondly, due to the hierarchical structure of the network, the proposed feature pyramid matching scheme conveniently realizes multi-scale anomaly detection.
2 STFPM
The schematic diagram of STFPM is as follows:
The core goal of STFPM is to train a network (ie, student network) so that it can encode the distribution of normal data. To achieve this goal, we designed two twin networks, one called "teacher" and the other called "student". The teacher network is initialized with pre-trained weights and is equipped with extensive knowledge. The student network is randomly initialized.
During the training process, normal data passes through both the teacher network and the student network. Through the loss function, feature distillation is performed on the pyramid layer feature map between the teacher network and the student network, so that the student network can learn the expression of normal data.
During the inference phase, test data is fed into the “teacher” and “student” networks. For normal data, the performance of the two should be very close. However, when faced with abnormal data, the "teacher" network can handle it well, while the "student" network gets confused. By comparing the differences in pyramid layer features between the two, we can determine whether there is abnormal data.
3 Experiments
Deep learning paper: PyramidFlow: High-Resolution Defect Contrastive Localization using Pyramid Normalizing Flow and its PyTorch implementation
PyramidFlow: High-Resolution Defect Contrastive Localization using Pyramid Normalizing Flow
PDF: https://arxiv.org/pdf/2303.02595.pdf
PyTorch code: https://github.com/shanglianlm0525/CvPytorch
PyTorch code: https://github.com/shanglianlm0525/PyTorch-Networks
1 Overview
PyramidFlow draws on the idea of template matching type method and uses the embedding vector of normal samples as a template to make matching judgments with test samples. Its main innovation points mainly include three aspects:
- First, the embedding vector of the normal sample is used as a template to match the test sample. This method can detect anomalies more accurately.
- Secondly, PyramidFlow uses multi-scale and multi-sample feature fusion and mapping, which can more comprehensively capture anomaly information in images.
- Finally, it is worth noting that PyramidFlow achieves better results than other methods without using pre-trained models, which demonstrates its effectiveness and uniqueness on the anomaly detection task.
2 PyramidFlow
The PyramidFlow algorithm consists of two processes, training and evaluation. The training process is similar to Siamese networks, optimizing the model by minimizing the frequency differences in pairs of images. For the evaluation process, latent templates are obtained by performing inference on the entire training dataset, and then latent contrast and pyramid combinations are applied to obtain anomaly localization maps.
2-1 Invertible Pyramid
2-2 Pyramid Coupling Block
2-3 Pyramid Normalizing Flow
3 Experiment
