"Natural Image Stitching with the Global Similarity Prior" the paper notes (a)
[[email protected]] @ (sinbad)
process steps of the recording method of image alignment grid summary based on the optimization algorithm for future memory.
VS global homography mesh optimization
2D image stitching is often done purely rotational motion by a camera or a photographing scene to a plane provided on the assumption that the scene model homography approximation, to obtain the mapping relationship between the images. When using only the global image alignment homography often enough. After another out of the image alignment grid-based optimization method after 2011. Wherein, the APAP method utilizes a plurality of local homography to achieve accurate alignment, but in only a partial constrained transformation image homography likely to cause image distortion so that the image quality deteriorates. GSP similarity transformation method is proposed the idea of priority, the camera motion model estimation using a reasonable exploded 2D scale factor and the rotation angle, so that the image distortion constraints, performance can be obtained more natural image mosaic. Combined OPTIMIZES local similarity to solve the more general problem of precise alignment of the image. In practice, the most widely used in both directions it should be on the phone panoramic image stitching and more fire recently video image stabilization. I guess before panorama stitching are based on stitching algorithm opencv as the prototype, a reference paper for the Automatic Panoramic Image Stitching using Invariant Features panorama stitching based on a global homography transformation, then the overlapping boundaries pyramid fusion, you can refer to google native camera on the phone . I do not know which phone from the beginning, panorama stitching began to be redefined, Internet search provider panorama stitching algorithm TOP class - the most famous Japanese morpho mesh optimization scheme is estimated to be used.
Algorithmic process
- [] Feature detection and matching
- [] FIG matching verification image twenty-two
- [] The method of generating a matching point the APAP
- [] And the estimated focal length 3-dimensional rotation angle estimation
- [] Scale and rotation angle estimation
- [] Mesh optimization
- [] Synthesis of texture mapping result in FIG.
About GSP algorithm
ALGORITHM mainly deformed grid image, an aligned manner by the features of the overlap region bound by the global similarity smooth non-overlapping regions, Also in the local similarity constraint, to ensure reduced distortion Nature mosaic image. In this chapter show optimized energy from the grid function analysis, each configured for optimized energy function specific analysis items, to facilitate understanding.
Total energy cost function for the articles:
wherein, Ψ_a (V) in alignment error term, Ψ_l (V) is a partial similar items, Ψ_g (V) for the global similarity item, λ_l is a partial weight parameters similar items.
For the two mutually overlapping image I_1, I_2, the image grid and using the vector
represents the coordinates of the vertices of the grid I, wherein m is the number of vertices. Two images can be expressed as vertex coordinates
align items:
We know that most of them will think of alignment feature matching, but here instead of using the coordinates of feature point matching point after the match but APAP paper proposed alignment as input. The effect shown below.
Tips: with respect to the distribution of feature points in the image, matching points on a lot uniformity. matching points corresponding to the vertices of the grid, (?) is particularly quad grid or mesh vertices have a closer look at the center point of the code. Need to look at how to generate presentation of papers APAP. The core idea is based on the image grid, each grid independent solving local homography moment H_l. Independent partial moment homography solver for solving similar methods with a single global homography moment, just before single-atopic individual global moment coefficient plus a moment, so that a single global homography solution method is called the DLT moment, local solving H_l is called the Moving DLT. details solving both placed behind the article.
Align items expressed in principle bound to find an online comparison of two graphs illustrate the image, but a bit out.
Further reference will be described.
两者的区别在于第一个是当求解I_1图像变形后的网格时，设I_1网格做warp得到的新的feature point坐标为未知，I_2网格做warp后feature point坐标点为测量值计算最小误差。同理反过来求解I_2图像变形后的网格。由于优化的是网格顶点（matching point）,因此，将特征点表达成其所有网格的四个顶点的双边线性组合：p=∑(k=1)^4▒〖a_k v_k 〗。其中，a_k(k=1,2,3,4)代表各自所占的面积比重，例如：p越靠近v_2远离v_3，则v_2越大，而a_3越小。而变换后的特征点表达为：p ̂=∑(k=1)^4▒〖a_k v ̂_k 〗，其中a_k(k=1,2,3,4)变换前后保持不变。
求解方法上，对齐项可以简化为A_a V ̂=0求解线性方程组。A_a表示在顶点集V ̂下的雅可比矩阵。所以A_a为其所在网格4个顶点差值权值矩阵。
补充说明1.
*1.单个全局单应性
图像q到p的单应性近似模型表达为
矩阵表达形式为
对公式进行展开可以得到两个线性方程，如下公式
对公式进行直接线性变换（DLT, Direct Linear Transformation）,将带估计的单应性矩阵H转变成变量向量h=〖(h_1 h_2… h_9)〗^T,同时将匹配特征点对p与q的已知变量关系转换已知线性变量矩阵
因而可以得到如下公式
通常，在两幅图像图像上可以有着上百或者上千对的匹配特征点对。因而，实际上在求解h时通常是通过使得累积平方和误差最小来评估（最小二乘法），如下公式所示。
此处，上式是一个超定方程的最小二乘解（当n>4时）。其利用‖h‖=1来限制h的自由度为8，其中A=[a_1^T a_2^T… a_i^T… a_n^T ]^T。其表示所有a_i (i=1,2,…,n)的纵向排列，大小为2n×9。
通常，利用SVD分解A矩阵，则可以很容易地获得最小二乘解h ̂，其值为最小奇异值对应的特征向量。事实上，为了避免误匹配带来的影响，通常也可以利用RANSAC结合单应性评估的四点法（最小采样四个匹配特征点对，通常为了保证高置信度，就仅采样四个匹配特征点对），来求解问题。
*2. 多个局部单应性
为了获得多个局部单应性，首先将图像进行网格划分。然后，根据每对匹配特征点对距离当前网格中心点的远近来进行加权，从而评估加权平方和误差最小，如下公式所示：
进而
m表示网格个数，需要评估m个局部单应性h ̂^((k) ) (k=1,2,…,m)。矩阵W^((k))是权重w_i^((k) ) (i=1,2,…,n)的对角组合
另外，权重
其中，点（x^((k) )，y^((k) )）表示查询图上第k个网格的中心点，点（x_i，y_i）表示查询图像上第i个特征点，参数σ是高斯函数的尺度因子，参数γ是一个阈值参数。该公式表明：一、特征点越靠近网格中心，则其对当前网格单应性估计的贡献越大；二、由于实际中多数特征点都是远离网格中心的，其权重趋近于0，为了避免这些特征点的贡献消失，设置阈值参数γ用于确立最小权重不低于该阈值。
另外，可以看到：假如γ=1，则w_i^((k) )=1，W^((k))=I退化成单位矩阵，于是，所有的局部单应性估计结果都会退化成全局单应性。从这个角度来看，阈值参数γ代表着局部单应性往全局单应性的趋近程度。