Reprinted from ( https://www.cnblogs.com/silence-hust/p/4192363.html )
1. Overview
Image fusion is an important part of image processing, which can collaboratively utilize multiple sensor image information of the same scene to output a Fusion images that are more suitable for human visual perception or further computer processing and analysis. It can significantly improve the shortcomings of a single sensor, improve the clarity and information content of the resulting image, and help to obtain more accurate, reliable, and comprehensive information about the target or scene.
Image fusion is mainly used in military and national defense, remote sensing, medical image processing, robotics, security and monitoring, biological monitoring and other fields. The fusion of infrared and visible light is used more and more maturely, displaying various information on one image and highlighting the target.
The fusion process can be carried out at different levels, which can be divided into: signal level, pixel level, feature level, and decision level.
1.1. Signal level
At the lowest level, the unprocessed sensor outputs are mixed in the signal domain to produce a fused signal. The fused signal has the same form but better quality as the source signal, and the signal from the sensor can be modeled as a random variable mixed with different correlated noise. In this case, fusion can be considered as an estimation process. To a large extent, signal-level image fusion is a problem of optimal concentration or distribution detection of signals, and has the highest requirements for signal registration in time and space.
1.2. Pixel-level
pixel-level image fusion is the most basic fusion among the three levels. The image obtained after pixel-level image fusion has more detailed information, such as the extraction of edges and textures, which is conducive to further analysis, processing and processing of images. Understanding, it can also expose potential targets, which is beneficial to judge and identify potential target pixels. This method can save as much information in the source image as possible, so that the fused picture has both content and details. In addition, this advantage is unique and only exists in pixel-level fusion. However, the limitations of pixel-level image fusion cannot be ignored. Because it operates on pixels, the computer has to process a large amount of data, and the processing time will be relatively long, so the fusion cannot be timely. After the image is displayed, real-time processing cannot be realized; in addition, during data communication, the amount of information is large, and it is easily affected by noise; and if the image is not strictly registered, it will directly participate in image fusion, which will lead to fusion. The image is blurry, and objects and details are unclear and imprecise.
1.3. Feature-level
feature-level image fusion is to extract feature information from the source image. These feature information are the target or interested areas in the source image, such as edges, people, buildings or vehicles. The feature information is analyzed, processed and integrated to obtain the fused image features. The accuracy of target recognition for the fused features is significantly higher than that of the original image. Feature-level fusion compresses the image information, and then analyzes and processes it with a computer, which consumes less memory and time than the pixel level, and improves the real-time performance of the required image. Feature-level image fusion does not have as high a requirement for image matching accuracy as the first layer, and the calculation speed is faster than the first layer, but it extracts image features as fusion information, so many detailed features will be lost.
1.4. Decision-making level
Decision-making image fusion is a method based on cognition. It is not only the highest level of image fusion method, but also the highest level of abstraction. Decision-level image fusion is targeted. According to the specific requirements of the question, the feature information obtained from the feature-level image is used, and then according to certain criteria and the credibility of each decision (the probability of the existence of the target) Make optimal decisions directly. Among the three fusion levels, the calculation amount of decision-level image fusion is the smallest, but this method has a strong dependence on the previous level, and the obtained image is not very clear compared with the previous two fusion methods. It is more difficult to implement decision-level image fusion, but the image is transmitted with minimal noise.