Comparative Study of Different Image Enhancement Methods for Detection and Classification of Strip Surface Defects
Based on the steel surface defect library, a variety of defect detection and classification experiments were carried out, and the results and cause analysis of image processing using convolutional neural network, Top-Hat transformation, edge detection, etc. were compared and analyzed. On the basis of directly using convolutional neural network classification, the images are processed in different ways, and the effect of deep learning on images processed in different ways under the same model is re-evaluated.
Compare image processing methods: three morphological Top-Hat transformations (multi-scale, equal proportion, adaptive), image sharpening (Sobel operator, Prewitt operator, Robert operator)
PS. The experiment is for the convenience of cross-use of Matlab and Tensorflow
1. Directly use image data for deep learning
There is no dirty data in the image in the defect library, and the convolutional neural network is directly used to classify the defect image without image enhancement. The initial model (Figure 1) was trained 10 times. From the loss curve (Figure 2) and accuracy curve (Figure 3), it can be seen that the model has been over-fitted in the 6th time, and the actual verification accuracy of the model is only 0.7506 when the training result of the 5th time is selected. 
It can be seen from the results that if the defect pictures are directly used for learning, the effect of local features learned by the neural network is not obvious, and similar defect types cannot be distinguished. Improving classification requires image augmentation of the defect library.
2. Image enhancement
image analysis
Use matlab for image histogram analysis:
select one image for each of the eleven types of defect images for histogram analysis. From their grayscale histograms, it can be known that the defect image is greatly affected by the background, and the grayscale distribution of the defect part and the background part has no clear main peak. Wanting to improve the image quality is to enhance the information of the defective part. It is planned to use gray-level layering technology to enhance images in three ways: pseudo-color image processing, morphological top-hat transformation, and image sharpening. However, observing the grayscale histogram, because there is no clear separation between the two peaks, the use of grayscale layering will lead to loss of defect information. Therefore, this paper mainly performs morphological top-hat transformation on the image to increase contrast and image sharpening to improve image quality and compare them.
Morphological top-hat transformation
Top-hat image enhancement method for uneven illumination. Mathematical morphology is to describe and analyze images at the level of collection, focusing on the geometric structure of images. Top-Hat (Top-Hat), also known as "top-hat" operation, the result image is the difference between the original image and its opening operation.
Since the opening operation first corrodes and then expands, it will remove the noise on the background color or the burrs and bumps on the contour line, and will also reduce the range of the foreground contour of the image. Therefore, the top hat can be used to separate lines or noise points that are brighter than adjacent areas. When an image has a large background (such as a black background), the top hat operation can be used to extract bright points or lines under the dark background.
Here, multi-scale morphological image enhancement, equal-scale morphological image enhancement and adaptive morphological image enhancement are performed on various types of images. (There are too many pictures, just put two)
As far as the three kinds of morphological processing are concerned, the multi-scale top-hat transformation is more universal for images. Multi-scale morphological bright top-hat transform and dark top-hat transform extract details for image enhancement. This method uses multi-scale top-hat transformation to extract the bright and dark detail features of the image at multiple scales, and according to the importance of local detail features at multiple scales, this method effectively enhances the contrast of the image, and can suppress noise amplification, and the visual effect is better .
image sharpening
Using the image based on Roberts operator, Sobel operator, Prewitt operator to extract the edge of the image respectively , and superimpose the extracted image with the original image to achieve image sharpening.
Sobel operator is one of the most important operators in pixel image edge detection. It is a discrete first-order difference operator used to calculate the approximate value of the first-order gradient of the image brightness function. Using this operator at any point in the image will generate the gradient vector corresponding to that point or its normal vector.
The Roberts operator is an operator that uses the local difference operator to find the edge. The edge of the image processed by the Robert operator is not very smooth. After analysis, because the Robert operator usually produces a wider response in the area near the edge of the image, the edge image detected by the above operator often needs to be thinned, and the accuracy of edge location is not very high .
The Prewitt operator is an edge detection of a first-order differential operator . It uses the gray level difference between the upper and lower pixels and the left and right adjacent points to reach the extreme value at the edge to detect the edge, remove part of the false edge, and have a smoothing effect on the noise . The principle is to use two direction templates to perform neighborhood convolution with the image in the image space. One of the two direction templates detects horizontal edges and the other detects vertical edges.
Three kinds of operators are used to extract the edges of various types of images. The experimental results are shown in the figure below (there are too many ten category images, so just a few). According
to the extraction effects of the three operators, it can be found that the prewitt operator has the best response to the image edge and the best image sharpening effect.
3. Deep learning after image enhancement
After processing the image in two modes of top-hat transformation and image sharpening, it can be seen that the two processing methods of multi-scale top-hat transformation and prewitt operator respectively have the best processing effect on the image in this mode. In order for the neural network to better learn image features, use the pretwitt operator to sharpen all images, and then use the initial convolutional neural network model to train the same number of times. The resulting loss curve and accuracy curve are shown in the figure below: It can be seen that when the same model is trained for the same number of times, the neural network can better extract local features of the image from the sharpened image, and the model effect
has been greatly improved .
Summarize
Morphological top-hat transformation:
In terms of the three morphological processes of multi-scale morphological image enhancement, equal-scale morphological image enhancement and adaptive morphological image enhancement, multi-scale top-hat transformation is more universal for images. Multi-scale morphological bright top-hat transform and dark top-hat transform extract details for image enhancement. This method uses multi-scale top-hat transformation to extract the bright and dark detail features of the image at multiple scales, and according to the importance of local detail features at multiple scales, this method effectively enhances the contrast of the image, and can suppress noise amplification, and the visual effect is better .
Image sharpening:
The Sobel operator is a discrete first-order difference operator, which is used to calculate the approximate value of the first-order gradient of the image brightness function. The Roberts edge detection operator is an operator that uses the local difference operator to find the edge. The edge of the image processed by the Robert operator is not very smooth. Since the Robert operator usually produces a wider response in the area near the edge of the image, the edge image detected by the above operator often needs to be thinned, and the accuracy of edge location is not very high. The Prewitt operator is an edge detection of a first-order differential operator. It uses the gray level difference between the upper and lower pixels and the left and right adjacent points to reach the extreme value at the edge to detect the edge, remove part of the false edge, and have a smoothing effect on the noise. It is done in the image space by using two directional templates to perform neighborhood convolution with the image. One of the two directional templates detects horizontal edges and the other detects vertical edges.
Because there is noise on the surface of the defect image, the gray level is close, there are abrupt changes in bright and abrupt changes, and the features are expressed in all directions. If the sobel operator is used, it will smooth the features of defects such as pits. Therefore, the prewitt operator is used to process the image, and the sharpening effect is better while suppressing noise.
In addition: This article is about doing some experiments after I just learned DIP at that time, and then I handed in homework as a class. Just write and write, if you want the source code, you can private message me. For convenience, I used Tensorflow and Matlab at the same time, but I forgot where to put the batch processing code later ƪ(˘⌣˘)ʃ
All the codes have been found and uploaded to github: https://github.com/Mintyt/Steel-surface-defect