Image stitching ( image stitching ) refers to the technology of stitching two or more overlapping images into a panorama or high-resolution image. There are two steps in image stitching:
1. Image alignment
Usually, the feature points of the image are obtained first , and then the alignment is completed through image registration .
2. Image mixing ( blending )
Explanation of basic terms:
Image alignment: Image alignment refers to finding the transformation relationship between two images, such as translation, rotation, and scaling, so that the same parts of the two images can overlap after transformation. The transformation relationship between images can be represented by a matrix. The more unknowns in the required matrix, the more complicated the relationship between the two images and the more difficult it is to align them. For example, the translation matrix has two unknowns, the affine transformation matrix has six unknowns, and the generalized projection matrix has eight unknowns. In order to simplify the problem, when taking the image, we will try to avoid translation, rotation or zooming at the same time, so as to simplify the difficulty of alignment. Usually, the feature points of the image are obtained first, and then the alignment is completed through image registration .
Feature point detection: Feature point detection refers to the method of finding feature points in an image by using information such as brightness, color, and gradient of the image. In image alignment, feature point detection can be used to obtain the feature points of two images, and then the alignment can be completed by pairing these feature points. Common feature point detection methods include: Harris Corner Detection ( Harris Corner Detection ) , scale invariant feature transformation ( SIFT ) , etc. Feature points are usually the corners of objects in the image, or places where the color changes greatly. However, because the overlapping parts of the two images to be aligned are limited, there are not many feature points in common, and most of the feature points are outliers ( outliers ), so image registration technology is required to complete the correspondence of feature points.
Image registration: refers to the technique of transforming the images to maximize the similarity between the two images. The method of calculating the similarity varies depending on the algorithm. For example, the root mean square difference of the two images can be used as an indicator of the similarity . The smaller the root mean square difference, the more similar the images are. In image alignment, image registration can complete the correspondence of feature points between images by maximizing the similarity of overlapping parts of images. The Random Sampling Consensus Algorithm ( RANSAC ) is one of the most common algorithms for image registration. Through recursion and sampling, the most appropriate transformation is found. The Random Sampling Consensus Algorithm is not a deterministic algorithm, but its advantage is that it can be used in many Among the outliers, find a reasonable transformation for the inner group value, which is suitable for image registration problems with many outliers.
Image blending: Due to factors such as parallax, lens distortion, and exposure differences, two aligned images may still have obvious boundaries rather than a complete composite image. Image blending refers to making the aligned images smooth Splicing technology [2] . Common methods include alpha blending , gradient -domain stitching [ 3 ] , etc.
panorama rectangle
The stitched panorama may become skewed or distorted due to cylindrical projection or the image is not completely level when shooting. Algorithm calculations can be used to adjust the stitched panorama back to a rectangle with a small amount of distortion and cropping