Medical image registration review study
At present, there are few reviews on medical image registration. The author only found two articles:
one was published in "Machine Vision and Applications"-Deep Learning in Medical Image Registration: A Survey and the
other was published in "Physics in Medicine and Biology" 》-- Deep Learning in Medical Image Registration: A Review
Other review papers on deep learning in mediacal image anlysis:
1, J. Ker, LP Wang, J. Rao, and T. Lim. Deep learning applications in medical image analysis. Ieee Access, 6:9375-9389, 2018. ISSN 2169-3536. doi: 10.1109/Access.2017.2788044.
2, G. Litjens , T. Kooi, BE Bejnordi, AAA Setio, F. Ciompi, M. Ghafoorian, JAWM van der Laak, B. van Ginneken, and CI Sanchez. A survey on deep learning in medical image analysis. Medical Image Analysis, 42:60–88, 2017. ISSN 1361-8415. doi: 10.1016/j.media.2017.07.005.
3、Y. Liu, X. Chen, Z. F.Wang, Z. J.Wang, R. K.Ward, and X. S.Wang. Deep learning for pixel-level image fusion: Recent advances and future prospects. Information Fusion, 42:158–173, 2018. ISSN 1566-2535. doi: 10.1016/j.inffus.2017.10.007.
4、A. Maier, C. Syben, T. Lasser, and C. Riess. A gentle introduction to deep learning in medical image processing. Zeitschrift Fur Medizinische Physik, 29(2):86–101, 2019. ISSN 0939-3889. doi: 10.1016/j.zemedi.2018.12.003.
5、P. Meyer, V. Noblet, C. Mazzara, and A. Lallement. Survey on deep learning for radiotherapy. Computers in Biology and Medicine, 98:126–146, 2018. ISSN 0010-4825. doi: 10.1016/j. compbiomed.2018.05.018.
6. B. Sahiner, A. Pezeshk, LM Hadjiiski, XS Wang, K. Drukker, KH Cha, RM Summers, and ML Giger. Deep learning in medical imaging and radiation therapy. Medical Physics, 46(1): e1--e36 , 2019. ISSN 0094-2405. doi: 10.1002/mp.13264.
7, Dinggang Shen, Guorong Wu, and Heung-Il Suk. Deep learning in medical image analysis. Annual review of biomedical engineering, 19:221–248, 2017 . ISSN 1545-42741523-9829. doi: 10.1146/annurev-bioeng-071516-044442.
8, ZW Zhang and E. Sejdic. Radiological images and machine learning: Trends, perspectives, and prospects. Computers in Biology and Medicine, 108: 354–370, 2019. ISSN 0010-4825. doi: 10.1016/j.compbiomed.2019.02.017.
PMB is relatively new and comprehensive, so the following is only a summary of the content of the paper.
Since the field of computer vision is updated very quickly, the learning summary information is relatively lagging. It is recommended to directly look at the latest conference CV journals and websites .
Review learning
Classification of medical image registration:
1. According to the input image angle: single-mode, multi-mode, different patients, same patients (same day, different days) registration.
2. According to the angle of the deformable model: rigid, affine, deformable.
3. According to the location: brain, lung, liver, etc.
4. According to the image dimensions: 2D-2D, 2D-3D, 3D-2D, 3D-3D registration.
5. According to the registration method: RL-based, Deep-similarity-based, supervised, unsupervised, GAN, registration validation using DL, others.
DIR: Deformable image registration.
Introduction to Deep Learning
1、CNN
2、U-net: U-net allows effective feature learning from a small number of training datasets
3、ResNet
4、DenseNet
5、RNN、LSTM
6、Reinforcement Learning (RL)：a type of machine learning that focused on predicting the best actions to take given its current state in an environment. Q-learning、Bellman equation.
7、GAN
Number of articles
Deep learning in medical image registration
DVF: Deformable vector field; SAM: Statistical appearance parameters; ADV: Adversarial; GT: Ground truth; DIR: Deformable image registration; TRE: Target registration error; RL: Reinforcement learning.
DL-based similarity measures
Traditional: intensity-based (SSD, MSD, (N)CC, (N)M, generally traditional similarity is only suitable for single-modal image registration, MI can be used for multi-modal registration, some variants of MI such as MIND effect It may be better.
DL-based similarity measures: No fixed form, the author has not yet fully understood the specific meaning, and will be added later. Some DL-baed similarity metrics are non-convex, and some strategies have been proposed to alleviate non-convex (such as optimizing a convex Supremacy).
RL in medical image registration
Combining RL and CNN, the image registration problem is decomposed into a series of classification tasks. That is, looking for a series of actions, such as rotation, translation, etc., to iteratively improve the level of image registration.
Rigid registration performed by most RL-based methods. RL-based method can improve the robustness of multi-modal registration.
Supervised & Unsupervised
The previous methods are all iterative, and the unsupervised and supervised DL-based methods are one-shot, so the registration time can be reduced. According to Figure 3, the current supervised and unsupervised methods account for more than half of the total, and they are increasing, and the hotspots of future research may also be these two.
Supervised : Ground truth is required. Generally can be divided into the following three categories:
(1) Random transformation: According to the aligned image pair, any given known rotation, translation and other deformations. The disadvantage is that the given deformation may not conform to the actual biomechanics and other conditions.
(2) Traditional registration-generated transformations: use traditional methods to generate DVF as GT.
(3) Model-based transformation generation: For a small number of known GT DVF image pairs, assuming that the distribution of DVF parameters is Gaussian, a large number of synthetic image pairs are generated.
The GT generation of FlowNet uses the above three methods.
Unsupervised : For example, VoxelMorph, which is more used in single-mode registration.
About the understanding of DVF Inverse Consistency: The author's understanding is that the registration should be round-trip. For a pair of unregistered images I1 and I2: the DVF of I1->I2 is f12, and the DVF of I2->I1 is f21, then f12·f21 = f21·f12=0, otherwise it is failure to trip back.
GAN in medical image registration
GAN network has two main functions in medical image registration:
(1) For multi-modal registration, you can first convert multi-modal data into single-modal data, that is, do an image translation as a pre-processing. Then the following process is similar to single-modal registration.
(2) Provide ADV loss.
doubt
It is mentioned many times in the article that image registration is a pathological problem, and the author has not yet understood it.