2017
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
code: https://paperswithcode.com/paper/what-uncertainties-do-we-need-in-bayesian
Abstract : We can model two kinds of uncertainty. Astronomical uncertainty captures the noise inherent in observations. Cognitive uncertainty, on the other hand, explains the uncertainty in the model—given enough data, the uncertainty can be explained. Traditionally, it has been difficult to model cognitive uncertainty in computer vision, but with new Bayesian deep learning tools, this is now possible. We investigate the benefits of modeling cognition and arbitrary uncertainty in Bayesian deep learning models of vision tasks. To this end, we propose a Bayesian deep learning framework that combines input-dependent arbitrary uncertainty and epistemic uncertainty. We investigate the model under the framework of per-pixel semantic segmentation and depth regression tasks. Furthermore, our explicit uncertainty formulation leads to new loss functions for these tasks, which can be explained as learning decay. This makes the loss more robust to noisy data and also provides new state-of-the-art results on segmentation and depth regression benchmarks.
2019
PHiSeg: Capturing Uncertainty in Medical Image Segmentation
code: https://github.com/baumgach/PHiSeg-code
Abstract : Segmentation of anatomy and pathology is inherently ambiguous. For example, structural boundaries may not be clearly visible, or different experts may have different annotation styles. Most current state-of-the-art methods do not account for this ambiguity, but instead learn a single mapping from image to segmentation. In this work, we propose a novel method to model the segmentation of a conditional probability distribution given an input image. We derive a hierarchical probabilistic model in which separate latent variables are responsible for modeling segmentation at different resolutions. Inference in this model can be performed efficiently using the variational autoencoder framework. We show that our proposed method can be used to generate more time, more realistic and diverse segmentation samples compared to recent related work.
2020
An Uncertainty-Driven GCN Refinement Strategy for Organ Segmentation
code: https://github.com/rodsom22/gcn_refinement
Abstract: Organ segmentation in CT volumes is an important preprocessing step in many computer-aided intervention and diagnostic methods. In recent years, convolutional neural networks have dominated the state of the art for this task. However, due to the high variability of organ shapes and similarities between tissues, this problem presents a challenging environment, and generating false-negative and false-positive regions in output segmentation is a common problem. Recent studies have shown that uncertainty analysis of this model can provide us with useful information about potential errors in segmentation. On this basis, we propose a segmentation refinement method based on uncertainty analysis and graph convolutional networks. We exploit the uncertainty level of a convolutional network in a given input volume to formulate a semi-supervised graph learning problem, which is solved by training a graph convolutional network. To test our method, we refine the initial output of a 2D U-Net. We validate our framework with the NIH's pancreas dataset and the medical segmentation decathlon's spleen dataset. We show that our method outperforms state-of-the-art CRF refinement methods, improving pancreas and spleen Dice scores by 1%, relative to the original predictions. Finally, we perform a sensitivity analysis on our proposed parameters and discuss the model's applicability to other CNN architectures, results, and current limitations to facilitate future work in research directions.
Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation
code: https://paperswithcode.com/paper/rectifying-pseudo-label-learning-via
Abstract: This paper investigates unsupervised domain adaptation to transfer knowledge from the source domain to the target domain in semantic segmentation. Existing methods are usually based on pseudo-labels to make full use of unlabeled target domain data. While the pseudo-labels of the target domain data are usually predicted by a model trained on the source domain. Therefore, due to the difference between the training and testing domains, the generated labels inevitably contain incorrect predictions, which can be transferred to the final adapted model, affecting the training process to a large extent. To overcome this problem, this paper proposes prediction uncertainty estimated explicitly during training to correct for unsupervised semantic segmentation adaptive pseudo-label learning. Given an input image, the model outputs a semantic segmentation prediction along with the uncertainty of the prediction. Specifically, we model uncertainty through forecast variance and incorporate uncertainty into the optimization objective. To verify the effectiveness of the method, we perform this on two popular synthetic-to-real semantic segmentation benchmarks, namely GTA5→Cityscape and Sync→Cityscape, and a cross-city benchmark, namely Cityscape→Oxford Robot Car. Through extensive experiments, we demonstrate that (1) the method dynamically sets different confidence thresholds depending on the prediction variance, (2) corrects for noisy pseudo-label learning, and (3) achieves significant improvements over traditional pseudo-label learning, across all Competitive performance was produced on all three benchmarks.
2021
Is segmentation uncertainty useful?
Abstract : Probabilistic image segmentation encodes different prediction confidences and ambiguities inherent in segmentation problems. While different probabilistic segmentation models have been designed to capture different aspects of uncertainty and ambiguity in segmentation, these modeling differences are rarely discussed in the context of uncertainty applications. We consider two common use cases for segmentation uncertainty, namely the assessment of segmentation quality and active learning. We consider four established probabilistic segmentation strategies, discuss their modeling capabilities, and investigate their performance on both tasks. We found that, for all models and both tasks, the returned uncertainty was positively correlated with segmentation error, but did not prove useful for active learning.
Uncertainty-Based Dynamic Graph Neighborhoods For Medical Segmentation
code: https://paperswithcode.com/paper/uncertainty-based-dynamic-graph-neighborhoods
Abstract : In recent years, deep learning-based methods have achieved success in basic medical image analysis tasks such as segmentation. Postprocessing and refining segmentation results is a common practice to reduce misclassifications produced by segmentation networks. In addition to widely used methods, such as conditional random fields (CRFs), which focus on the structure of segmented volumes/regions, a recent graph-based method exploits certain and uncertain points in the graph and is based on small graph convolutional networks ( GCN) to refine the segmentation. However, this method suffers from two drawbacks: most of the edges in the graph are randomly assigned, and the GCN is trained independently of the segmentation network. To address these issues, we define a new neighbor selection mechanism based on feature distances and combine the two networks during training. Based on experimental results on pancreatic segmentation from computed tomography (CT) images, we demonstrate improvements in quantitative measures. Furthermore, examining the dynamic neighbors created by our method, edges between semantically similar image parts are observed. The proposed method also shows qualitative enhancement on segmentation maps, as shown in the visual results.
UACANet: Uncertainty Augmented Context Attention for Polyp Segmentation
code: https://paperswithcode.com/paper/uacanet-uncertainty-augmented-context
Abstract : We propose Uncertainty Augmented Background Attention Network (UACANet) for polyp segmentation, which considers an uncertain region of saliency maps. We construct a modified U-Net shape network with encoder and dechizer, and compute saliency maps in each bottom-up flow prediction module and propagate to the next prediction module. In each prediction module, foreground, background, and uncertainty region maps are computed using previously predicted saliency maps, and the three region maps represented by each are aggregated. The relationship between each representation in the feature map and each pixel in the feature map is then computed. We conduct experiments on five popular polyp segmentation benchmarks, Cvasir, CVC-ClinicDB, ETIS, CVC-ColonDB, and CVC-300, and our method achieves state-of-the-art performance. In particular, we achieve an average dice of 76.6% on the ETIS dataset, a 13.8% improvement over previous state-of-the-art methods.
