[Python image processing] Wavelet transform performs image fusion

Wavelet Transform Performs Image Fusion

Image Fusion Basics

Image fusion is the process of combining multiple input images into a single output image that contains a better description of the scene than a single input image. A good image fusion method should have the following properties:

It can preserve most useful information on different images
It produces no artifacts that interfere with or mislead the visual appearance of the image or subsequent image processing steps
it must be robust
It should not discard any salient information in the input image

Perform image fusion using wavelet transform

In this section, we will learn how to use to pywtperform image fusion operations (wavelet coefficients are fused).

(1) First import the required Pythonlibrary :

import pywt
import cv2
import numpy as np
import numpy as np
import matplotlib.pylab as plt

(2) Define the function fuseCoeff()and define the coefficient fusion according to the fusion method:

def fuseCoeff(cooef1, cooef2, method):
    if (method == 'mean'):
        cooef = (cooef1 + cooef2) / 2
    elif (method == 'min'):
        cooef = np.minimum(cooef1,cooef2)
    elif (method == 'max'):
        cooef = np.maximum(cooef1,cooef2)
    else:
        cooef = []

    return cooef

(3) Specify the fusion method (can be minor meanetc.):

fusion_method = 'mean'

(4) Read the input image to be fused. We'll use two different versions of the same original image, the first with the left part blurred and the second with the right part blurred. Our goal is to blend the details in these two images with each other:

im1 = cv2.imread('4.jpg',0)
im2 = cv2.imread('5.jpg',0)

(5) Our two images are the same size, so we need to resize the images to have the same size:

im2 = cv2.resize(im2,(im1.shape[1], im1.shape[0]))

image fusion

(1) Perform wavelet transform on each input image and calculate the corresponding coefficients:

wavelet = 'sym2' #'bior1.1' #'haar' #'db1'
cooef1 = pywt.wavedec2(im1[:,:], wavelet)
cooef2 = pywt.wavedec2(im2[:,:], wavelet)

(2) For each level in the two images, perform fusion according to the desired options:

fused_cooef = []
for i in range(len(cooef1)):
    if(i == 0):
        fused_cooef.append(fuseCoeff(cooef1[0], cooef2[0], fusion_method))
    else:
        c1 = fuseCoeff(cooef1[i][0], cooef2[i][0],fusion_method)
        c2 = fuseCoeff(cooef1[i][1], cooef2[i][1], fusion_method)
        c3 = fuseCoeff(cooef1[i][2], cooef2[i][2], fusion_method)
        fused_cooef.append((c1,c2,c3))

(3) After fusing the coefficients, we need to use IDWTto transfer back to get the image:

#print(len(fused_cooef))
fused_image = pywt.waverec2(fused_cooef, wavelet)

(4) Normalize the output value to be within unit8the data range:

fused_image = 255*fused_image / np.max(fused_image)
fused_image = fused_image.astype(np.uint8)

(5) Finally, draw the original image, average image and wavelet fused image as follows:

plt.figure(figsize=(20,20))
plt.gray()
plt.subplot(221), plt.imshow(im1), plt.axis('off'), plt.title('Image1', size=10)
plt.subplot(222), plt.imshow(im2), plt.axis('off'), plt.title('Image2', size=10)
plt.subplot(223), plt.imshow(im1//2 + im2// 2), plt.axis('off'), plt.title('Average Image', size=10) #cv2.cvtColor(fused_image,cv2.COLOR_BGR2RGB))

plt.subplot(224), plt.imshow(fused_image), plt.axis('off'), plt.title('Fused Image with Wavelets', size=10) #cv2.cvtColor(fused_image,cv2.COLOR_BGR2RGB))
plt.tight_layout()
plt.show()

The resulting image looks like this:

image fusion

From the above result images, it can be seen that compared with the average image output, the output image obtained by image fusion using wavelet is richer in details (use PSNRas evaluation criterion).