FPGA ISP pyramid fusion -Round1

FPGA ISP pyramid fusion -Round1

Recently received a task to achieve wide dynamic Gaussian pyramid and Laplacian pyramid integration in FPGA, ...

But think about it, calm state of mind, I still own to learn the algorithm. Regardless of the final result FPGA implementation, but also as a way to improve their own kind.

Thank you codes CSshengxy available on Github, such as a night light to illuminate me. Interested students have to go search download, must give his star ah!

First, look at the code from the main start

clear all;
clc;

% leftImage = double(imread('apple.png'));
% rightImage = double(imread('orange.png'));

iternum = 5;

if (size(leftImage) ~= size(rightImage))
    error('Input images are not the same size!')
end

% 预处理，使图片height,width为偶数
[rows, cols, channels] = size(leftImage);

% mask gaussian
mask = double(zeros(rows, cols, channels));
mask(:, 1:floor(cols/2), :) = ones(rows, floor(cols/2), channels);
mask_pyramid = GaussianPyramid(mask, iternum);


% leftImage pyramid and rightImage pyramid
left_pyramid = LaplacianPyramid(leftImage, iternum);
right_pyramid = LaplacianPyramid(rightImage, iternum);

% TODO: get blend laplacian pyramid
blend_pyramid = cell(iternum, 1);
for i = 1:iternum
    blend_pyramid{i} = left_pyramid{i} .* mask_pyramid{i} + right_pyramid{i} .* (1 - mask_pyramid{i});
end

% reconstruct the blend image
blendImage = LaplacianReconstruct(blend_pyramid);
imwrite(uint8(blendImage), 'blendImage.png');

figure;
imshow(uint8(leftImage));
title('leftImage');
figure;
imshow(uint8(rightImage));
title('rightImage');
figure;
imshow('blendImage.png');
title('blendImage');

First, the main function reads two pictures, is divided into left and right, size determination of the two images are the same

And image preprocessing to ensure that the image resolution is even.

Each layer mask needs to be saved here, as the weight, multiplied by the Laplacian image pyramid of the corresponding layer, and then superimposed.

Today's focus is to take over, Gaussian pyramid, namely the mask pyramid.

function pyramid=GaussianPyramid(image, iternum)
    gaussian_filter = [1, 4, 6, 4, 1]*(1/16);
    gaussian_filter = gaussian_filter' * gaussian_filter;
    
    pyramid = cell(iternum, 1);
    pyramid{1} = image;
    
    for i = 2 : iternum
        % gaussian filtering
        % imfilter 对于rgb图像相当于三个通道分别处理,然后进行cat操作
        temp = imfilter(pyramid{i-1}, gaussian_filter, 'replicate');
        % down sampling
        rows = size(temp, 1);
        cols = size(temp, 2);
        temp = temp(1:2:rows, 1:2:cols, :);
        pyramid{i} = temp;
        
    end

5:00 OF first calculates the Gaussian distribution in the horizontal direction, and then calculate the two 5x5 Gaussian kernel

Here the bottom of the pyramid, the image that is input directly assigned to the cellular array pyramid {1}, the key is to see how the second layer 2 to layer iternum calculated.

For the first i-layer, first i-1 th layer is subjected to a Gaussian filter, where the boundary by way of copying processing (later by simulation, determining a boundary processing, impact algorithm's, FPGA when implementing the image algorithm, I usually not treated).

Results then the horizontal and vertical directions according to output odd column extraction, to obtain the right level corresponding to the weight of the current i and stored cellular array.

Note here fused initial weight design, as shown, the left side of all 1, all 0 right.

Why such a design, you can download the code runs know, ha ha!

But when the actual completion of the pyramid-based HDR, two images of the right weight to see in the paper, will take into account the edge of the image, saturation, brightness. After this I learned of this foundation, try to change their own code to achieve what HDR fusion of heavy weights calculated.