Medical image registration
0, target:
0.1 Summary of content
The purpose of this paper's research on registration is to use registration as a kind of preprocessing , not as a research on innovative points. So it involves more traditional registration methods. It mainly includes the following aspects.
1. Understand what registration is? What is the purpose of medical image registration? What are the registration methods?
2. Understand the nouns, MNI, atlas, registration template and other professional terms
3. What are the public registration libraries antspy, registration toolbox, etc.
4. How to perform the registration operation? How to register within the same brain modality?
1. What is registration?
1.1 What is registration?
Image registration Image registration is a typical problem and technical difficulty in the field of image processing research. Its purpose is to compare or fuse images obtained for the same object under different conditions. For example, the images will come from different acquisition equipment and taken from different Time , different shooting angles , etc. Sometimes it is also necessary to use image registration issues for different objects.
Specifically, for two images in a set of image data sets, one image (floating image, moving image) is mapped to another image (reference image, fixed image) by looking for a spatial transformation, so that the two Points in the figure corresponding to the same position in space are corresponding one-to-one to achieve the purpose of information fusion.
Medical image registration:
In these two pictures, it is easy to see that they are not aligned, and the left side is tilted relative to the right side. Although our human visual system can still connect the corresponding anatomical relationships together, it is not true when we use computer analysis. were able.
Therefore, we want the same anatomical structures to be aligned, expressed in terms of coordinates, we want them to be at the same coordinates. As shown in the figure below
, the process of aligning these anatomical structures is called registration.
Therefore, the purpose of registration is to make the same anatomical structures in the same position for easy analysis. CT and MRI can be registered to see changes in the same structure in the two modalities, making it easier to diagnose the disease. Similarly, PET-MRI, etc. can also be registered. Different MRI sequences can also be registered. For the same MRI sequence, two images taken before and after can also be registered, and so on.
Note: In fact, registration is to solve the different sizes of different data scales, and it is a standardized operation for quantitative analysis.
Note: If the brain is to be registered, it is best to remove the skull and other tissues first. Because we only want to register the brain tissue, we don't care about whether the nose and eyes are registered, and they will interfere with the speed and accuracy of registration.
1.2 Classification of registration
According to different criteria, registration methods can be classified in the following ways:
1.2.1 According to spatial dimensions
If only the spatial dimension is considered, it can be divided into 2D/2D, 2D/3D, and 3D/3D. If time series factors are considered, there is also the problem of registering two images extracted at different times.
1.2.2 According to the characteristics and similarity measure based on the algorithm
- Based on internal features
Internal features refer to information extracted from the interior of the image itself:- Based on feature points: a set of feature points that have special geometric significance and can be located (such as discontinuous points, turning points of graphics, line intersections, etc.). In medical images, they can even be points with anatomical significance.
- Surface-based: Use segmentation method to extract the outline of the part of interest (curve or surface) as a feature space for comparison [4] [5] .
- Based on pixel values: Use the pixels or voxels of the entire image (Intensity-Based) to form the feature space. Calculating similarity measures based on the statistical information of pixel values can be divided into least squares method , Fourier method, cross-correlation method, mutual information method, etc.
- Based on external features
In medical images, certain marker points on the image are obtained by fixing markers on the patient or injecting developing substances into the body, which are called external feature points.
1.2.3 According to transformation properties
Spatial transformation of images can be divided into rigid body transformation (rigid) and non-rigid body transformation (non-rigid, deformable). Usually there are rigid body transformation, affine transformation, projection transformation and curve transformation.
(1) Rigid body transformation : The so-called rigid body means that the distance between any two points inside the object remains unchanged. For example, the human brain can be viewed as a rigid body. When processing human brain images, rigid body transformation is often used for image registration in different directions [4]. Rigid body transformation can be decomposed into rotation and translation
(2) Affine transformation: Affine transformation [5] maps straight lines into straight lines and maintains parallelism. Specific manifestations can be uniform scale transformation with consistent scale transformation coefficients in all directions or non-uniform scale transformation and shear transformation with inconsistent transformation coefficients. Uniform scale transformation is mostly used for photographic images using lens systems. In this case, the image of the object is directly related to the distance between the object and the imaging optical instrument. General affine transformation can be used to correct the tilt of the CT gantry. Distortion caused by shear or MR gradient coil imperfections.
(3) Projective transformation : Similar to affine transformation, projective transformation [6] maps straight lines into straight lines, but no longer maintains the parallel property. Projection transformation is mainly used for registration of two-dimensional projection images and three-dimensional volume images.
(4) Nonlinear transformation : Nonlinear transformation [7] is also called curved transformation, which transforms straight lines into curves. Polynomial functions are commonly used, such as quadratic, cubic functions and thin plate spline functions. Exponential functions are also sometimes used. Nonlinear transformation is mostly used to deform anatomical atlases to fit image data or to register images of thoracic and abdominal organs with global deformation.
1.2.4 According to optimization algorithm
When the comparison feature takes the form of a set of feature points, the solution to the transformation can be found through a system of simultaneous equations. However, in general, the registration problem will be transformed into the problem of solving the optimal value of the similarity measure. In the calculation method, it is usually necessary to use appropriate iterative optimization algorithms, such as gradient descent method, Newton method, Powell method, genetic algorithm, etc.
1.3 Medical image registration
1.3.1 According to image modality
Since medical imaging equipment can provide different forms of images with different information about patients (computed tomography CT, magnetic resonance MRI, positron emission tomography PET, functional magnetic resonance fMRI, etc.), it can be divided into single- modal and Multi-modal.
(1) Monomodality medical image registration: means that the two images to be registered are obtained with the same imaging device.
(2) Multimodality medical image registration: means that the two images to be registered come from different imaging devices .
1.3.2 According to the subject
It can be divided into three types : Intrasubject (images from the same patient ) , Intersubject (from different patients ) and Atlas (registration of patient data and atlas).
(1) Self-image registration (intra-subject) :
The images to be registered can be of the same person and belong to the patient's own image registration (intra-subject). Images of the same organ or anatomical part obtained from the same patient at different times can be used for comparison to monitor the development of the disease and the treatment process. If there is no local tissue resection, this kind of registration can generally be achieved by rigid body transformation.
(2) Human image registration (inter-subject)
In addition, sometimes it is necessary to compare the subject's image with the image of the same part of a typical normal person to determine whether the subject is normal; if it is abnormal, it may also be compared with the image of the subject. Compare typical images of some diseases to determine whether patients belong to the same category. These all belong to image registration between different people (inter-subject) [10]. Due to differences in individual anatomy, registration of the latter is obviously more difficult than that of the former.
(3) Image registration with the Atlas Method (Atlas Method) or registration with the physical space.
Due to physiological differences between different people, the shape, size, and position of the same anatomical structure will be very different, which makes the image registration of different people difficult. The problem has become the biggest problem in medical image analysis today. When comparing and analyzing different medical images, it is difficult to accurately find the corresponding anatomical information. This requires a computerized standard atlas that details the various anatomical locations of the human body .
There are roughly two types of common methods: ** First, use a common standard to compare**. For example, to compare the PET or MR images of two patients [11], first, both images must be mapped to a Go to a common reference space, and then compare the two in this space. The most commonly used one is ** Talairach standard space **, which can compare different human brain images; ** The second is the nonlinear deformation method **, Imitating the elastic mechanics method, one person's 3D image is gradually transformed so that it can finally best match the 3D image of another person.
2 Professional terms and knowledge of medical image registration
MNI, atlas, registration template
2.1 Registration template:
When it comes to registration, we must first introduce the template. Due to individual differences in the human brain, the coordinates of the scanned images in space are also different. During research, individual differences must first be eliminated and the coordinates unified, that is, the subjects need to be All brains are "corrected/registered" to the standard template , so that subsequent statistical analysis can be performed.
2.2 Image space
There are three kinds of image spaces: standard space, structural space, and functional space.
Each space can have different resolutions. For example, the standard space template MNI152 has resolutions of 1mm, 2mm, and 0.5mm.
Templates of various resolutions and needs are provided. We can align our data to standard space as part of preprocessing.
https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/Atlases
Match images to standard space:
When using multiple data sets to train the network, each data set has a different resolution and size. And I want all datasets to have the same resolution and can specify its resolution how should I do this?
For example, the brain MRI data sets include Brats (240x240x155) and IXI (256x256xz, z=28-136). These two data sets must be used in the same network training, and the resolution and size must be unified.
The best way is to allocate them to the same standard space (such as MNI152)
Question:
Does MNI152 only have T1 data?
2.1 MNI space
At present, the most widely used standard template in the world is MNI152 . MNI is the abbreviation of Montreal Neurosciences Institute, Canada . MNI152 is obtained by weighted average of 152 3D T1 data of 152 healthy people. According to different averaging algorithms, There are two types: linear and nonlinear.
The above picture comes from: https://nist.mni.mcgill.ca/atlases/ , where 152 nonlinear 6th generation refers to high-dimensional nonlinear registration before weighted average MNI152 template.
Talairach brains were sectioned and photographed from the famous Talairach and Tournoux atlas. The map has corresponding broadly consistent markers for Brodmann brain regions.
MNI305 is the predecessor of MNI152. As the name suggests, it comes from MRI scan images of 305 healthy people. The international consortium for brain mapping (ICBM) adopted MNI305 as their template, which is the follow-up SMP99 standard template. Brain Imaging International Another template recommended by the alliance is ICBM152 . In neuroimaging research, most researchers use this template as a brain template.
Subsequently, ICBM, the International Association of Brain Imaging, launched a more representative template**: ICBM452**, which is the result of matching 452 human brains with ICBM305 through conversion, but the current use of ICBM452 is relatively small.
The general course of the brain imaging template can be summarized as:
MNI305 → MNI152 → ICBM152 → ICBM452
3. Related open source tools
3.1 Tool overview
Traditional classic tools:
Material Presented at ITK Tutorials :
https://www.kitware.com//courses-in-medical-image-analysis-that-use-itk/
MeVisLab :
https://www.mevislab.de/mevislab /
FSL: https://blog.csdn.net/u014264373/article/details/124852831
MATLAB:
- Image Registration techniques using MATLAB
- Registering Multimodal MRI Images using Matlab.
- elastix: a toolbox for rigid and nonrigid registration of images.
- niftyreg: a toolbox for doing near real-time robust rigid, affine (using block matching) and non-rigid image registration (using a refactored version of the free form deformation algorithm).
PYTHON:
- VoxelMorph: A Learning Framework for Deformable Medical Image Registration
- Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach
- AirLab: Autograd Image Registration Laboratory
Other:
https://paperswithcode.com/task/image-registration
3.2 Medical image registration software ANTs (Advanced Normalization Tools)
Since the brain shapes of different individuals are different, in order to compare the differences in brain structure and function between different individuals, it is necessary to first register the different individuals to a standard brain template. The registration process deforms the individual brain image so that the individual brain is as consistent as possible with the template image.
In fact, registration does not necessarily require a template. You can also find a patient's brain as a template, and all other patients will be matched to the same space as him.
3.2.1 Description of ANTs
Medical image registration software ANTs (Advanced Normalization Tools) is a medical image processing software based on C language and is relatively fast.
ANTs supports 2D and 3D images, including files in the following formats:
• Nifti (.nii, .nii.gz)
• Analyze (.hdr + .img / .img.gz)
• MetaImage (.mha)
• Other formats through itk ::ImageFileWriter / itk::ImageFileWriter such as jpg, tiff, etc. See ITK documentation.
ANTs official documentation: http://stnava.github.io/ANTsDoc/
3.2.2 Installation and use of ANTs
There are two main forms of ANTs installation:
one is the installation based on source code , and the corresponding function can be called directly on the command line after the installation is completed ;
the other is the installation based on python, that is, the antspy library. Call the corresponding package for use. The two methods currently only support Linux and Mac systems.
(1), Install ANTs based on source code
- First, install git, cmak and c++ compiler;
- Run from the command line:
git clone git://github.com/ANTsX/ANTs.git #从github上克隆相应的仓库，保存在当前目录下的ANTs文件夹下
mkdir antsbin #创建antsbin文件夹
cd antsbin #进入antsbin文件夹
ccmake ../ANTs #进入cmake界面，然后依次按下'C'键，稍作等待，再按下'C'键和'G'键，分别完成设置和生成后回到命令行
make -j 4 #进行编译，需要运行较长时间
If you encounter cmake or ccmake version mismatch, you need to uninstall and reinstall it, and update the environment variables.
- After compilation, if the bin directory appears in the antsbin directory, you can proceed to the next step. If there is no bin directory, you need to create the bin directory yourself and copy the files from the three places into it. The specific steps are as follows:
# 在 antsbin 目录下
mkdir bin #在antsbin下建立bin目录
cp ./ANTS-build/Examples/* ./bin # 将ANTS-build/Examples下的文件复制到bin目录中
cp ./staging/bin/* ./bin #将staging/bin下的文件复制到bin目录中
cp ../ANTs/Scripts/* ./bin #将ANTs/Scripts下的文件复制到bin目录中
- Set environment variables, change .bashrc or .profile files
cd ~ #回到home文件夹下
vi ~/.bashrc #打开vi进行编辑，按'i'进入插入模式，并在文档末尾插入以下内容
export ANTSPATH=/home/username/antsbin/bin/
export PATH=“$ANTSPATH:$PATH”
#以上路径要和真实路径一致，然后依次按'ESC'键,'Shift'+':'键，'w'键和'q'键，然后回车保存并退出
source ~/.bashrc #激活相应的环境配置
Use of ANTs
There are various .sh files under the ANTs/Scripts path. The more commonly used ones are antsRegistrationSyN.sh, etc. For convenience, you can add the path of the .sh file to the environment variable:
vi ~/.bashrc #打开.bashrc文件并在末尾添加以下内容
export PATH=$PATH:/home/username/ANTs/Scripts
#保存并退出
source ~/.bashrc #使环境变量生效
Then use antsRegistrationSyN.sh directly on the command line. If the usage method of this command is given, the configuration is successful. If an error message is given, the configuration fails.
Because I don’t have registration data in .nii format, I did the experiment with images in .jpg format. The fixed image (above picture) and moving image (below picture) used are as follows: The registration command is
:
antsRegistrationSyN.sh -d 2 -f fixed_img.jpg -m moving_img.jpg -o output
Among them, -d 2 indicates that the data is a 2-dimensional image, -f fixed_img.jpg is the image name corresponding to the fixed image, -m moving_img.jpg is the image name corresponding to the moving image, and -o output is the prefix name of the output result. The output data is as follows:
output0GenericAffine.mat, output1Warp.nii.gz represent the mapping relationship estimated by linear transformation and nonlinear transformation respectively, outputWarped.nii.gz represents the image after registering moving_img.jpg to fixed_img.jpg, outputInverseWarped. nii.gz represents the image after registering fixed_img.jpg to moving_img.jpg. The pictures of outputWarped.nii and outputInverseWarped.nii are as follows:
3.2.3
a, install antspy based on python
Please refer to the installation method:
https://github.com/ANTsX/ANTsPy
For MacOS and Linux:
Note: antspyx is used when installing
pip install antspyx
antspy should only support macos and linux systems now. Windows is not supported yet. You can also use Git to install:
git clone https://github.com/ANTsX/ANTsPy
cd ANTsPy
python3 setup.py install
The above installation method can only install version 0.1.4. When using this version, there will be some small bugs, such as: for the registration of move to fix, the int type must be converted to float type; and version 0.1.8 does not
exist For such a problem, the specific installation method is very simple and can be solved with one line of code:
pip install git+https://github.com/ANTsX/ANTsPy.git #可能不能一次性成功，数个小时差不多
b, antspy uses
How to use antspy can be found in the official user manual:
https://antspyx.readthedocs.io/en/latest/registration.html
If it is related to registration, you only need to look at the content corresponding to Core and Registration.
import ants
def registration(fix_path,move_path,save_path,label_path =None,save_label_path = None):
types = ['Translation', 'Rigid', 'Similarity', 'QuickRigid', 'DenseRigid', 'BOLDRigid', 'Affine', 'AffineFast', 'BOLDAffine',
'TRSAA', 'ElasticSyN', 'SyN', 'SyNRA', 'SyNOnly', 'SyNCC', 'SyNabp', 'SyNBold', 'SyNBoldAff', 'SyNAggro', 'TVMSQ']
fix_img = ants.image_read(fix_path)
move_img = ants.image_read(move_path)
outs = ants.registration(fix_img,move_img,type_of_transform=types[1])
reg_img = outs['warpedmovout']
ants.image_write(reg_img,save_path)
if label_path != None:
move_label_img = ants.image_read(move_path)
reg_label_img = ants.apply_transforms(fix_img,move_label_img,transformlist = out['fwdtransforms'],interpolator='nearestNeighbor')
ants.image_write(reg_label_img,save_label_path)
##将 move向 fix配准
fix_path = 'MNI152_T1_2mm_brain.nii.gz'
move_path = 'sub_strokecace0011_gaojinping_dwi_raw.nrrd'
save_path = 'reg_2min.nii.gz'
registration(fix_path,move_path,save_path)
The effect is shown in the figure: after registration, template, and original
Reference:
Overview of medical image registration technology - Miaozu's article - Zhihu https://zhuanlan.zhihu.com/p/267339046
[MRI multi-sequence, multi-center data set preprocessing – using FSL-Flirt to register data]
Original text Link: https://blog.csdn.net/u014264373/article/details/124852831
[Registration standard template for nuclear magnetic data processing]
https://blog.csdn.net/happyhorizion/article/details/79579453
[Image registration Review]
https://zhuanlan.zhihu.com/p/80985475
[Using ANTs for MRI (structure, function) image registration - installation, registration detailed answers]
https://blog.csdn.net/xj4math/article /details/120895684
[Installation and usage instructions of medical image registration software ANTs (Advanced Normalization Tools)]
https://blog.csdn.net/zuzhiang/article/details/104930000