The instructor asked me to do a small project on brain tumor segmentation. Start learning image segmentation and MRI related knowledge today! (md from classification to detection to segmentation, from remote sensing images to brain images, I am really grateful...angry)
data set
BraTS is a data set for the MICCAI brain tumor segmentation competition. The training set in BraTs 2018 has 285 cases.
Each case has four modalities (t1, t2, flair, t1ce) and needs to be divided into three parts: whole tumor (WT), enhance tumor (ET), and tumor core (TC), which is equivalent to three labels.
Each case contains 4 modal MRI sequences and 1 seg file. All sequence sizes are (240, 240, 155), as shown below:
MR image attribute information: (240x240x155) An MR sequence has 155 pictures, and the size of each picture is 240x240
HGG、LGG
The BraTs training set is divided into HGG and LGG, which respectively represent:
HGG: High-grade glioma (WHO grade 3 to 4) is a poorly differentiated glioma; these tumors are malignant tumors and the prognosis of patients is poor.
LGG: Low-grade glioma (WHO grade 1 to 2) is a well-differentiated glioma; although this type of tumor is not biologically benign, the prognosis of the patient is relatively good.
Domain shift domain transfer
refers to trying to apply the learned model to test data (target domain) with a different distribution from the training data (source domain), and its performance will decrease. Due to the multi-modality of medical images, the situation of data domain shift is more natural and severe. As shown in the figure below, different medical images (magnetic resonance (MR) imaging and computed tomography (CT)) can see that the heart area of the two images is visually distinct. There is no doubt that the network trained on MR images cannot be used for testing on CT images.
multimodal
Multimodality, in layman's terms, means that one thing can be expressed in many different forms. Personally, I understand that four modes are equivalent to channels. There are four modalities in the BraTs data set: t1, t2, t1ce, and flair. In the medical community, t1, t2, t1ce, and flair are called sequences. A case can have multiple sequences, and each sequence consists of many In addition, the way to obtain each sequence is different. For example, t1 and t2 are two different sequences generated due to different physical quantities of electromagnetic waves measured. Another example is that the t1ce sequence requires the blood to create a contrast agent before doing MR.
nii format data
The extension of the standard NIFTI image is .nii, which also includes header files and image data. The NIFTI format can also use independent image files (.img) and header files (.hdr).
The nii image is a three-dimensional image. After slicing, it represents
the vector plane, the Median sagittal section (the human body is cut from front to back),
the coronal plane (the human body is cut from left to right), and
the horizontal plane Transverse section.
How to read nii file
1.NII files can be opened directly with the software ITK-SNAP
2. Use code to analyze
2.1 Install torchio
pip install torchio
Read the file and display the picture (only t1 is read below):
import torchio as tio
t1_path = 'BraTS19_2013_2_1_t1.nii.gz'
t1_img = tio.ScalarImage(t1_path)
t1_img.plot()
t1 is displayed as follows:
2.2 Install nibabel:
pip install nibabel
Read the file and display the picture (only t1 is read below):
import nibabel as nib
import matplotlib.pyplot as plt
import numpy as np
img = nib.load('BraTS19_2013_2_1_flair.nii.gz')
print(img.shape) # shape(240, 240, 155)
print(img.header['db_name'])
width, height, queue = img.dataobj.shape # 由文件本身维度确定，可能是3维，也可能是4维
#print("width",width) # 240
#print("height",height) # 240
#print("queue",queue) # 155
nib.viewers.OrthoSlicer3D(img.dataobj).show()
num = 1
for i in range(0,queue,10):
img_arr = img.dataobj[:,:,i]
plt.subplot(5,4,num)
plt.imshow(img_arr,cmap='gray')
num +=1
plt.show()
The picture is displayed as follows: (If it is black, drag the picture with the mouse or move the wheel, the coordinates may be at the origin)
It seems that the second method will display 3D, which is more dynamic. If you just want to see what the data looks like, both methods can be used.
T1 weighted, T2 weighted
MRI scanning methods can be simply divided into two categories: conventional scanning and functional scanning. Conventional scans mainly reflect anatomy; functional scans reflect functional information such as human metabolism and blood flow in different ways.
T1 Weighted and T2 Weighted are the most commonly used and basic conventional scans. Almost all clinical MRI examinations include T1-weighted and T2-weighted scans. The "weighted" here means to highlight.
T1 looks at the structure
The overall sense of the T1 image is very close to the "customary color matching style" of "clinical images". You see white matter is white, gray matter is gray, and cerebrospinal fluid is black. So T1 images can see various tomographic anatomy diagrams. So This is where the saying “T1 looks at the anatomy” comes from.
T2 looks at lesions
The T2 signal is related to the water content (and Flair is bound to water). The T2 signal of many lesions is stronger than the surrounding normal tissue. And many lesions are accompanied by tissue edema. As you can see from the picture below, the very white ones are edema, and then the whiter ones are The shadow is about the lesion (red).
As shown in the figure below, left: T1-weighted image, right: T2-weighted image,
T1-weighted highlights anatomical structures, while T2-weighted can highlight lesions.
T1CE
The t1ce sequence requires the blood to be used to create a contrast agent (pigment) before doing MR. The bright areas have rich blood supply , and the enhancement clearly shows that the blood flow is very rich. Where does the blood flow need to be fast? Tumors ~ they need a lot of energy to continue to divide Nutrition.
As shown in the picture below, the blue area is an enhancing tumor, and the green ones surrounded by it are gangrene (ju) (necrotic components). Gangrene is cell necrosis and then liquefaction. So these gangrenes are also called non- Non-enhancing tumor.
Flair
FLAIR sequence is a commonly used sequence in nuclear magnetic resonance (MR). Its full name is liquid attenuated inversion recovery sequence, also known as water suppression imaging technology. In layman's terms, it is a pressure water image. On this sequence, the cerebrospinal fluid shows low signal (darker), and solid lesions and lesions containing bound water show obvious high signal (brighter).
Split parts WT, ET, TC
The picture below is a sequence of cases from the Brats data set [the color labels have nothing to do with the pictures above]. I imported the flair sequence and the corresponding segmentation labels through ITK-SNAP. There are three parts that need to be segmented, namely WT, ET, and TC.
The data set stipulates that the green in the picture is the edema area (ED, peritumoral edema) (label 2), the yellow is the enhancing tumor area (ET, enhancing tumor) (label 4), and the red is gangrene (NET, non-enhancing tumor) (label 1), background (label 0)
These labels are then merged into 3 nested sub-regions:
whole tumor (WT) -------- Contains all labels (red + yellow + green)
tumor core (TC) --------- Red+yellow label
enhancing tumor (ET) -------yellow label
即 WT = ED + ET + NET
TC = ET+NET
ET
Related
1. MICCAI+BraTS+multimodal+t1,t2,flair,t1c+HGG,LGG+WT,ET,TC
2. python reads .nii.gz file and displays medical images
3. How to read NIFTI format images