Compared with conventional natural images, the codec network represented by UNet is more widely used in medical image segmentation. Various common medical imaging methods, including computed tomography (CT), magnetic resonance imaging (Magnetic Resonance Imaging, MRI), ultrasound imaging (Ultrasound Imaging), X-ray imaging (X-ray Imaging) and optical coherence tomography Scanning (Optical Coherence Tomography, OCT), etc. Clinically, imaging examination is a very important diagnostic method. In the image examination of various modalities, the accurate segmentation of various organs and lesions is a key step in image analysis. At present, deep learning image segmentation is brilliant in various image detection and segmentation. For example, pulmonary nodule detection based on chest CT, glioma segmentation based on intracranial MR images, left ventricle segmentation based on cardiac CT, nodule detection based on thyroid ultrasound, and lung organ segmentation based on X-rays.
Although a series of encoding and decoding segmentation networks based on UNet have made great progress in various types of medical image segmentation, and some application designs based on related models have been widely used in clinical analysis. However, the complexity and diversity of medical imaging itself also greatly affect the generalization and versatility of the segmentation model, which is mainly reflected in the following aspects:
(1) There are large differences between medical images of various modalities, such as the size of the study cohort, image size and dimension, resolution, and voxel intensity.
(2) Extreme imbalance of semantic labels for segmentation. Compared with the normal tissue in the image, the lesion area generally only occupies a very small part, which results in an extreme abnormality between the voxel labels of the normal tissue and the voxel labels of the lesion tissue.
(3) There are large differences in expert annotation between different image data, and the annotation results of some images may be ambiguous.
(4) Some data sets have obvious differences in attributes such as image geometry and shape, and the problems of slice misalignment and anisotropy are also very serious.
For the above problems, the academic community believes that it is necessary to develop a general-purpose, transferable and strong generalization performance medical image segmentation model. In 2018, scholars in the field of medical images organized a 10-round medical image segmentation competition (The Medical Segmentation Decathlon, MSD), requiring participants to design a general-purpose segmentation model for 10 different medical image segmentation tasks, These tasks cover various typical challenges of medical image segmentation, including small sample problems, label imbalance problems, multi-center data distribution problems, and small target segmentation problems. As shown in Figure 1, the 10-athlon segmentation competition is divided into a development phase and a testing phase. The development phase includes tumor segmentation of brain MRI images, segmentation of the anterior and posterior parts of the hippocampus in brain MRI images, and segmentation of CT lung nodules. Seven tasks including segmentation, CT liver and liver cancer segmentation, CT pancreas and pancreatic cancer segmentation, MRI prostate peripheral zone and transformation zone segmentation, and cardiac MRI left atrium segmentation. The three tasks in the test stage are all CT imaging tasks, including Colon cancer lesion segmentation, liver vessel segmentation, and spleen segmentation. Figure 2 is an example of data and annotations for MRI brain tumor segmentation.
For the above segmentation tasks, relevant researchers believe that since the UNet structure was proposed, there have been many improved versions based on it, and designs such as residual connections, dense connections, attention mechanisms, deep supervision, and feature calibration have been introduced to it. Generally speaking, for specific For split tasks, the effect is limited, and sometimes even worse. Previously, the key point that everyone focused on was the design of the network structure, and little attention was paid to parameters other than the model. Based on this, the team that won the first place in this competition proposed a unified UNet framework: nnUNet, or no-new-Net, can be understood as not a new network structure design, which can automatically configure parameters and perform tasks according to data tasks. network training. The paper proposing nnUNet is nnU-Net: Breaking the Spell on Successful Medical Image Segmentation, and nnUNet achieved SOTA-level results in the general medical image segmentation task at that time.
Compared with the previous various UNet improvements, nnUNet pays more attention to the image preprocessing work, can automatically judge the image modality and perform the corresponding normalization operation, and perform different image voxel spacing according to the cubic spline interpolation. Re-sampling. In terms of network structure, nnUNet proposes 3 networks based on the original UNet, namely 2D UNet, 3D UNet and 2 cascaded 3D UNets. The first is used to generate coarse segmentation results, and the second is used to refine coarse segmentation results. The segmentation results are shown in Figure 3. The 3 UNet models can be configured, designed and trained independently of each other. Compared with the original UNet, nnUNet has fine-tuned its structure: using padded convolution to achieve the same output and input shape, using Leaky ReLU instead of ReLU, using Instance normalization instead of batch normalization. nnUNet can automatically set hyperparameters, such as training batch size, image block size, number of downsampling, etc. All UNet architectures have passed five-fold cross-validation, using cross-entropy loss and Dice loss as the loss function during training, the optimizer uses Adam, and a learning rate dynamic adjustment strategy is set, and an online data enhancement strategy is also used during training.
Although nnUNet does not propose a new structural design, it breaks the previous research routines in the field of medical image segmentation, and unifies the medical image segmentation task framework through some designs outside the network structure, making it extremely versatile. nnUNet pays more attention to parameter configurations other than the network structure, including image preprocessing, training design, model integration during inference, and postprocessing. nnUNet source code can refer to:
https://github.com/MIC-DKFZ/nnunet
According to readers' feedback on this series, the subsequent relevant content will be gradually updated to the GitHub address of Deep Learning Semantic Segmentation and Practical Guide:
https://github.com/luwill/Semantic-Segmentation-Guide
Past highlights:
Intensive reading of deep learning papers [6]: UNet++
Intensive reading of deep learning papers [5]: Attention UNet
Intensive reading of deep learning papers [4]: RefineNet
Intensive reading of deep learning papers [3]: SegNet
Intensive reading of deep learning papers [2]: UNet network
Intensive reading of deep learning papers [1]: FCN full convolutional network
Explainer video is here! Machine learning formula derivation and code implementation open record!
end! "Machine Learning Formula Derivation and Code Implementation" PPT download of chapters 1-26