This blog post mainly analyzes the cell segmentation algorithm based on traditional computer vision and the cell segmentation algorithm based on deep learning. Mainly for medical image segmentation algorithms. Including the cell segmentation effect of FCN full convolutional network, U-Net cell segmentation effect. Consulted dozens of papers using deep learning to segment medical images, and summarized them.
Medical image segmentation-analysis and summary of cell segmentation algorithms
1. Summary of cell segmentation based on traditional algorithms
1.1 Several commonly used binarization methods
1.1.1 Global binarization is simple, but it depends on the selected threshold. It has poor processing effect on images with uneven lighting, and images with similar background and foreground.
1.1.2 Optimal threshold method (based on histogram). This method is based on the probability distribution of the image pixel intensity, and the junction of the two probability distributions is selected as the threshold to obtain the smallest segmentation error.
1.1.3 OTSU Otsu threshold method (based on clustering). This method is based on statistical discriminant analysis to calculate a threshold that can minimize the intra-class variance and maximize the intra-class variance.
1.2 Watershed algorithm
1.2.1 Watershed algorithm based on marker point constraints. Generally for cell image segmentation, there are markers in the cell image. The markers are used as the initial position of a dam, and water is continuously injected until the water levels of the two dams intersect. The dam is repaired and the segmentation is completed.
1.3 Hybrid method
In order to improve segmentation performance, a combination of multiple methods is often used to alleviate the performance degradation caused by a single method.
1.3.1 Use graph cut method for cell segmentation.
Using the method of mixed graph cutting, the image is automatically binarized, and then the LoG (Laplacian-of-Gaussian) constrained by the automatic selection method based on the distance map is used for seed point extraction.
1.4 Minimize the model for cell segmentation
Model-based methods for cell detection and segmentation often rely excessively on the prior information of cell shape features. Due to the well-defined nature of the cell, this will introduce deviations in cell segmentation. However, the contour method does not depend on the shape of the cell nucleus and the staining characteristics, and can handle most of the segmentation problems in medical cell imaging. The minimum model segmentation method is as follows:
(1) Detect all closed contours; (2) contour evaluation; (3) generate non-overlapping segmentation; (4) contour optimization; (5) separate concave targets; (6) classify the remaining areas, based on the characteristics of dyeing, Divided into target cells and other objects.
2. Summary of cell segmentation algorithms based on deep learning
2.1 FCN Fully Convolutional Neural Network
FCN classifies images at the pixel level, thereby solving the problem of semantic segmentation. Unlike the classic CNN that uses a fully connected layer to obtain a fixed-length feature vector for classification (fully connected layer + softmax output) after the convolutional layer, FCN can accept input images of any size, and use a deconvolutional layer for the last convolutional layer The feature map is up-sampled to restore it to the same size of the input image, so that a prediction can be generated for each pixel, while retaining the spatial information in the original input image, and finally the up-sampled feature map is step-by-step Pixel classification.
2.2 U-Net
The network architecture is as follows:
The U-Net model is an improved FCN structure, named after its structure resembles the letter U, and is used in the semantic segmentation of medical images. It consists of the contracting path on the left half and the expansive path on the right half. The compression channel is a typical convolutional neural network structure. It repeats the structure of 2 convolutional layers and 1 maximum pooling layer. After each pooling operation, the dimensionality of the feature map is doubled. In the expansion channel, first perform a deconvolution operation to reduce the dimensionality of the feature map by half, and then splice the feature maps obtained from the corresponding compression channel to reconstitute a feature map of 2 times the size, and then use 2 convolutional layers Perform feature extraction and repeat this structure. In the final output layer, two convolutional layers are used to map the 64-dimensional feature map into a 2-dimensional output map.
3. Implementation of cell segmentation based on traditional algorithms
3.1 Implementation steps of traditional segmentation algorithm
In order to evaluate the performance of traditional segmentation algorithms, cell segmentation is implemented on a given data set. The following are the implementation steps. (Because it is a cooperative project, the algorithm is not convenient to disclose).
3.2 Result analysis
Cell segmentation based on traditional computer vision can efficiently process cells with large differences in the foreground and background and large distances between the targets. However, when the foreground and background are similar, and the cells are connected together, the effect is Poor. If you need to improve the accuracy, you need to use curvature and local feature constraints.
4. Display of the effect of cell segmentation algorithm based on deep learning
Some of the following images are researched deep learning-based cell segmentation renderings, including the segmentation results of various medical images. Through various analyses and comparisons, it can be seen that some algorithms still fail to separate close cells.
The first column in the above figure is the original image, the second column is Ground Truth, and the third to fifth columns are various segmentation effects based on deep learning models. It can be seen that, compared with the true value, the segmentation effect is not good for the cells that are closer, and it is easy to split into one piece.
The above figure is a segmentation effect diagram based on U-Net and its related variants. It can be seen that the segmentation effect is better. Analyzing the reasons, it can be seen that in the original image, each cell is very scattered, not connected together, and the foreground and background are quite different.
By observing the effect diagram, it can be seen that the application of deep learning algorithm divides a cell connected together into one cell.
For intensive cell segmentation, the accuracy of the segmentation algorithm based on deep learning is still low. The place marked by the red box in the picture above is a large piece of cells connected together.
The above figure is to observe the result of segmentation after zooming in the local area. The left image is Ground Truth, and the right image is the segmentation result image.
The figure above is the result of cell segmentation using FCN. The first column is the original image, the second column is Ground Truth, and the third column is the segmentation result. It can be seen from the place marked in the red box that in the original image, the closer image is segmented by the deep learning model and then connected into one piece.
The result shown here is the same as the above image. The position marked by the red box shows that deep learning segmentation has encountered the same problems as traditional computer vision segmentation.
The above methods show some improved U-Net algorithms. For the general U-Net, it is easy to divide the closely-attached cells into a whole, and some optimization methods and deep learning model adjustments can be used to separate some difficult samples. .
In the above picture, the segmentation effect is better when the cells are widely distributed. As for the closely connected cells, it can be seen from the effect diagram indicated by the yellow arrow that U-Net is still not divided, and the model using the optimization method can divide such cells.
For dense cells, the segmentation effect is only about 80%.
As can be seen from the red box in the figure above, for cells that are very similar to the background color, deep learning methods cannot be effectively segmented.
Some U-Net applications in cell segmentation and 3D reconstruction.
5. Comparison and summary of traditional cell segmentation algorithms and cell segmentation algorithms based on deep learning
Through the display of various actual segmentation effects given in Section 4, it can be seen that for the large difference in the foreground and background of the cells, and the cells are relatively independent, there is no large-area contact, and the situation where they are stuck together, deep learning can be better. Segmentation. And when dealing with a piece of cell that is difficult to separate, the segmentation algorithm based on deep learning cannot separate well. Among the evaluation criteria used in cell segmentation, the performance of most segmentation algorithms is around 90%. For the currently given data set, due to the small difference between the foreground and the background and the many cells connected together, the segmentation algorithm based on deep learning can not guarantee the segmentation effect compared with the traditional algorithm, which is greatly improved.
For the data set problem, due to the high resolution of the cell image, there are about 300 cells in an image, and each contour of the cell needs to be manually labeled, which is time-consuming and expensive. Therefore, self-made data sets require a large annotation cost.