This article was first published on the WeChat public account CVHub. Private reprinting or selling to other platforms is strictly prohibited, and offenders will be held accountable.
Title: Self-aware and Cross-sample Prototypical Learning for Semi-supervised Medical Image Segmentation
Paper: arxiv.org/pdf/2305.16…
guide
This paper introduces a new method for semi-supervised medical image segmentation . In medical image segmentation, due to limited labeled data and abundant unlabeled data, consistency learning plays an important role in utilizing unlabeled data while effectively utilizing limited labeled data. However, the author believes that the existing research often ignores the influence of the following two factors on consistent learning, namely:
- predictive diversity
- training stability
Meanwhile, limited annotated data are often insufficient for constructing pseudo-labels of intra-class closeness and inter-class difference .
To address these issues, the paper proposes a self-aware and cross-sample prototype learning method ( SCP-Net), which enhances the diversity of predictions in consensus learning by exploiting broader semantic information derived from multiple inputs. Furthermore, a self-aware consistency learning method is introduced to improve the closeness of pseudo-labels within each category using unlabeled data. In addition, the authors integrated the double loss reweighting method into the cross-sample prototype consistency learning method, which improved the reliability and stability of the model.
Ultimately, extensive experiments on ACDCand PROMISE12datasets demonstrate SCP-Netsuperiority over other state-of-the-art semi-supervised segmentation methods and achieve significant performance gains over limited supervised training.
motivation
We briefly mentioned above that this paper mainly enhances the diversity of prediction results in consistent learning by introducing self-awareness and cross-sample category prototypes. Traditional methods use global prototypes, while self-aware and cross-sampleSCP-Net prototypes are used to generate two different prototype prediction results to enhance semantic information interaction and ensure inconsistency in consistent training.
At the same time, the paper leverages the prediction uncertainty between self-aware prototype predictions and multiple prediction results to reweight the consistency constraint loss across sample prototypes. By doing so, the adverse effect of label noise in challenging regions such as low-contrast regions or adherent edges can be reduced, leading to a more stable and consistent training process, improving the performance and accuracy of the model.
method
It can be seen that this framework is a very typical U-Netarchitecture, and the core point is the effective use of data, so we only need to care about SPCCand CPCCmodules.
SPCC
The self-aware prototype consistency constraint ( SPCC) aims to enhance the intra-class closeness of segmentation predictions. It utilizes self-aware prototype predictions as pseudo-labels for supervision. By using self-aware prototypes, the method enables more accurate alignment of features on the samples themselves, ensuring intra-class consistency of predictions. By introducing this constraint, the model can better learn the common features within the category and improve the segmentation performance.
CPCC
The cross-sample-aware prototype consistency constraint ( CPCC) aims to gain reliable knowledge from other training samples. It employs a double weighting approach to adjust the contribution of constraints by taking into account uncertainty estimates and self-aware probability predictions.
First, uncertainty estimation is used to reduce the influence of suspicious pseudo-labels, reducing the weight of these regions with large uncertainties in training. Second, the self-aware probability prediction is introduced, and the maximum value on a specific category is calculated as the weight to further enhance CPCCthe reliability. Through this double weighting method, CPCCthe prototype prediction across samples can be better utilized to improve the performance and stability of the segmentation model.
Loss
通过上述两个约束模块的引入能够增强预测的多样性和训练的有效性，并减轻噪声预测的负面影响，从而提高半监督分割的性能。最后，我们可以引申出最终的约束表达式：
总的来说，SCP-Net的损失函数由监督损失和无监督一致性损失的组合，其中监督损失是有交叉熵损失函数和Dice损失函数组合而成的复合函数构成，用于监督标记数据的训练过程。另一方面，对于标记数据和未标记数据，作者利用
和
提供无监督的一致性约束，用于网络训练并探索有价值的未标记知识。
此外，这里还使用到一个时间依赖的高斯预热函数的权重，用于平衡监督损失和无监督损失。通过使用这个权重函数，可以在训练过程中逐渐平衡监督损失和无监督损失的贡献。
实验
对于ACDC数据集，与有限监督基线相比，SCP-Net在RV、Myo和LV的Dice相似系数（DSC）上分别提高了7.02％、6.13％和6.32％，并且在DSC和平均表面距离（ASSD）上与完全监督基线相当。与其他方法相比，SCP-Net在DSC和ASSD上取得最佳效果，分别比第二名高出1.58％和0.24。此外，通过ACDC数据集的可视化示例，SCP-Net展示了对RV、Myo和LV类别的一致且准确的分割结果，证明了无监督的原型一致性约束对于提取有价值的未标记信息以改善分割性能的有效性。
For the experimental results on prostate segmentation, SCP-Net outperforms the limited supervision baseline at 10% labeling ratio and achieves an improvement of 16.18% on the DSC metric and 10.35 on the ASSD metric. The DSC of SCP-Net reaches 77.06%, which is 5.63% higher than the second place CCT. These improved results show that SPCC and CPCC have a positive effect on exploiting unlabeled information.
The effectiveness of the SCPNet design can be seen through ablation experiments. The design of SPCC and CPCC facilitates the improvement of semi-supervised segmentation performance, and improves the reliability and stability of consensus training through the integration of prediction uncertainty and self-aware confidence.
Summarize
This paper presents a novel approach SCP-Netto improve consensus learning methods in semi-supervised medical image segmentation through self-awareness and cross-sample prototype learning. Experimental results show that the method can achieve better segmentation performance than other state-of-the-art semi-supervised methods when using limited labeled data.
CVHub is a high-quality knowledge sharing platform focusing on the field of computer vision. The original rate of technical articles on the whole site reaches 99%. It presents you with comprehensive, multi-field, in-depth cutting-edge AI paper solutions and supporting industry-level application solutions every day. Solution, provide scientific research | technology | employment one-stop service, covering supervised/semi-supervised/unsupervised/self-supervised various 2D/3D detection/classification/segmentation/tracking/pose/super-resolution/reconstruction and other full-stack fields And generative models such as the latest AIGC. Pay attention to the WeChat public account, welcome to participate in real-time academic & technical interactive exchanges, receive a CV learning spree, and subscribe to the latest information on school recruitment & social recruitment of major domestic and foreign companies!