Table of contents

Image registration and stitching are important tasks in the fields of computer vision and image processing, which allow multiple images to be aligned and fused into a complete image. Especially in fields such as drones and medical images, image registration and stitching are crucial to obtain high-quality results. The image registration and stitching algorithm based on the Scale-Invariant Feature Transform (SIFT) operator is a commonly used method, which can effectively deal with the scale, rotation and perspective changes of the image.

1. Mathematical formula

The SIFT operator realizes image registration and stitching by detecting key points (feature points) in the image and calculating their local feature descriptors. The mathematical representation of keypoint detection is as follows:

Gaussian Difference Pyramid: $D(x, y, \sigma) = (G(x, y, k\sigma) - G(x, y, \sigma)) * I(x, y)$ Among them, $G
( x, y, \sigma)$ is the Gaussian blur function, $k$ is the scale factor, and $I(x, y)$ is the image.

Scale-space extremum detection: Detect extreme points in the Gaussian difference pyramid as key points.

Orientation Assignment: Calculate the main orientation of keypoints, which is used to generate descriptors.

Feature descriptor: According to the gradient direction around the key point, a local feature descriptor is generated for subsequent matching and registration.

2. Implementation process

The implementation process of image registration and mosaic algorithm based on SIFT operator can be summarized as follows:

Image preprocessing: Preprocessing the images to be registered or stitched, including denoising, scale transformation, etc.

Key point detection: Use the SIFT operator to detect key points in the preprocessed image, and obtain the coordinates and scale information of the key points.

Feature Descriptor Computation: Compute the local feature descriptor for each keypoint.

Keypoint matching: To match the keypoints of two images, nearest neighbor matching or other methods can be used.

Registration and stitching: According to the matching key points, image registration and stitching operations are performed.

3. Application fields

The image registration and mosaic algorithm based on SIFT operator is widely used in many application fields, including but not limited to:

Remote sensing image processing: registration and stitching of images acquired by multiple satellites or drones to generate a larger map or scene.

Medical image processing: In medical images, images acquired at different times or by different imaging devices can be registered for disease monitoring and analysis.

Computer-Aided Design: Stitching multiple images into one larger image for modeling, design, and visualization.

Document Scanning and Merging: Merge multiple document images into one large image for document management and archiving.

4. Core program

.......................................................................
%Use Ransac to get good amount of inliners.
inliners=RANSAC(img_ref,image_cores);
inliners1=RANSAC(img_ref1,image_cores1);
%Find inliner points for the image
[final_inliner_ref,final_inliner_img]=InlierPoint(img_ref,image_cores,inliners);
[final_inliner_ref1,final_inliner_img1]=InlierPoint(img_ref1,image_cores1,inliners1);

%Plot new inliner correspondence
PlotImageCores(final_inliner_ref,final_inliner_img,img1);
PlotImageCores(final_inliner_ref1,final_inliner_img1,img3);
%re-compute the homography using the points.
H_re=computeH(final_inliner_ref,final_inliner_img);
H_re1=computeH(final_inliner_ref1,final_inliner_img1);
%reproject on the image and do the image stitching 
final_mosaic=Mosaicing(ref_img2,img1,H_re);
final_mosaic=uint8(final_mosaic);
final_mosaic1=Mosaicing(ref_img2,img3,H_re1);
final_mosaic1=uint8(final_mosaic1);
final_image=Mosaicing(final_mosaic,final_mosaic1,eye(3));
[rowF,colF,K]=size(final_mosaic);
final_image=zeros(rowF,colF,3);
for i=1:rowF
    for j=1:colF
        for k=1:K
            if final_mosaic(i,j,k)>final_mosaic1(i,j,k)
                final_image(i,j,k)=final_mosaic(i,j,k);
            elseif final_mosaic(i,j,k)<final_mosaic1(i,j,k)
                final_image(i,j,k)=final_mosaic1(i,j,k);
            elseif final_mosaic(i,j,k)==final_mosaic1(i,j,k)
                final_image(i,j,k)=final_mosaic(i,j,k);
            end
        end
    end
end
figure;
imshow(uint8(final_image),'InitialMagnification','fit');
title('全景图');

for i=1:3
    % 高斯函数的频域低通滤波，以提高图像光滑度
    
    % 生成sigma=200的高斯函数
    ff = creat_gauss(final_image(:,:,i),300);
    
    % 频域低通滤波
    out = lowpass_freq_filt(final_image(:,:,i), ff);
    final_image(:,:,i)=out;
end

figure;
imshow(uint8(final_image),'InitialMagnification','fit');
title('频域低通滤波后的全景图');
up2186

5. Simulation results

6. References

Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91-110.

Brown, M., & Lowe, D. G. (2007). Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 74(1), 59-73.

Ma, J., Luo, X., Xu, M., & Wang, S. (2017). A survey of keypoint-based image stitching algorithms. The Visual Computer, 33(12), 1593-1604.

Walas, K., & Forczmański, P. (2019). A review of image stitching algorithms for handheld digital cameras. Journal of Real-Time Image Processing, 16(6), 2201-2220.

Yu, L., Zhu, X., & Gao, H. (2020). Key technologies and applications of image stitching: a survey. Multimedia Systems, 26(4), 393-414.

Matlab simulation of image registration and mosaic algorithm based on sift operator