A review of research on the application of U-Net and its variants in medical image segmentation
Paper from: Chinese Journal of Biomedical Engineering 2022
Abstract: Medical image segmentation can provide reliable basis for clinical diagnosis, treatment and pathological research, and can assist doctors to make accurate judgments on patients' conditions.The emergence of segmentation networks based on deep learning solves the problems of traditional automatic segmentation methods such as weak robustness and low accuracy.. U-Net stands out among many segmentation networks due to its excellent performance. Researchers have successively proposed a variety of improved variants based on U-Net. Taking the U-Net network and its variants as the main line, we first introduce the network structure and common improvement methods of U-Net in detail; then according to the different segmentation objects, the U-Net variant network is further divided into general-purpose segmentation networks and specific segmentation networks. segmentation network, and discusses its research progress in medical image segmentation; finally, the difficulties and problems existing in the current research work are analyzed, and the future development direction is prospected.
Mainly based on Unet variant research
Basic introduction to UNet model
The UNet model proposed in 2015 is what we learnAn excellent model that must be learned for semantic segmentation, which is both lightweight and high-performance, so it is usually used as the baseline test model for semantic segmentation tasks, and this is still the case today, which shows its excellence. UNet is essentially a fully convolutional neural network model. Its name comes from its architectural shape: the model as a whole presents a "U" shape. its birth isIn order to solve the problem of semantic segmentation of medical images,, but the development in the following years has also confirmed that it is an all-round player in semantic segmentation tasks. Perhaps this is the excellence of excellent network architecture.
Perhaps the following three points can explain why UNet performs outstandingly in medical imaging:
1.UNet's U-shaped network structure densely integrates shallow features and deep features；
2. The amount of medical imaging data matches the size of the UNet model, effectively avoiding overfitting.；
3. The structure of medical images is simple and fixed, with low semantic information。
U-Net network improvement
Although U-Net has made a major breakthrough in medical image segmentation, there are still problemsNetwork scalability is not ideal、It is easy to overfit when training small data setsdeficiencies in other aspects. Depending on factors such as the characteristics of the target object, image modality, and application scenarios, researchers usually select appropriate improvement methods based on the original network to improve the performance of network segmentation.
Directions for improvement
1. Modify jump connection
Jump connectionNot only can it integrate high- and low-level features, but it can also help speed up network convergence., the combination of different forms of skip connections allows the network to achieve ideal segmentation results without additional back-end processing.
2. Introduce new technologies
Propose new attention modules, fusion modules, etc.
3. Integrate other networks
Combine resNet with U-Net and other fusion networks.
4. Network cascading
Medical image segmentation based on U-Net and its improved variants
Summary and Outlook
This article reviews the application of U-Net and improved networks based on U-Net in medical image segmentation. Research progress shows that U-Net and its variant networks have broad application prospects in medical image segmentation. The segmentation results are better than traditional methods. However, it should also be noted thatproblems：
1) Compared with other semantic segmentation data sets, the scale of the medical image data set is limited due to the patient privacy involved.Sampling is more difficult and therefore smaller. andThe U-Net network has a large number of parameters and is prone to overfitting when the data set is small., so it is necessary to choose an appropriate improvement method to adjust the network structure.
２)== Medical image imaging is more complex than general images, and artifacts may exist==, thus affecting the segmentation effect. Appropriate preprocessing algorithms need to be selected. to reduce the impact of artifacts on segmentation.
３） There are many targets in medical images that require refined segmentation such as blood vessels and cells. Optimization methods such as dense connections and attention mechanisms can be used to highlight features that are beneficial to the segmentation task.。
4) Although the above segmentation model has reached a high level in terms of accuracy, Iou and DSC, it is based on laboratory data after all. Deep learning models such as U-Net still face many difficulties before they can be used in actual diagnosis and treatment environments. For example, the data used for training and testing are carefully selected, and these image data are too "clean";Deep learning models are highly dependent on image annotation, but even experienced annotators will make subjective errors during the annotation process.. These problems will have a potential impact on the accuracy of the learning model.
5) Many network models have complex structures and huge parameters, which may be difficult to reproduce in actual application environments. Therefore, the future direction of work should be to make appropriate adjustments to the model while ensuring high accuracy in the laboratory, so that deep learning can move from theory to clinical application. The future development of U-Net will be based on maintaining the encoding-decoding structure and
skip connections, combined with other neural networks to further improve the accuracy of medical image segmentation, such as:
1) Graph neural network (GAN) can generate higher quality images, which has been experimentally proven , using GAN to generate additional training images to train the network can improve segmentation performance to a certain extent. Therefore, before using U-Net or its variant network segmentation, using GAN to generate corresponding medical images to expand the training set can solve the problem of small data set size to a certain extent. Generative Adversarial Network
2) (GNN) has natural advantages when processing 3D data. There have been experiments using gated graph neural networks to achieve interactive 3D medical image segmentation. Therefore, U-Net can be combined with GNN to better extract 3D features;