题目

A Generic Camera Model and Calibration Method for Conventional, Wide-Angle, and Fish-Eye Lenses
常规、广角和鱼眼镜头的通用相机模型和校准方法

下载地址： https://www.researchgate.net/publication/6899685_A_Generic_Camera_Model_and_Calibration_Method_for_Conventional_Wide-Angle_and_Fish-Eye_Lenses

Abstract

Fish-eye lenses are convenient in such applications where a very wide angle of view is needed but their use for measurement purposes has been limited by the lack of an accurate, generic, and easy-to-use calibration procedure. We hence propose a generic camera model, which is suitable for fish-eye lens cameras as well as for conventional and wide-angle lens cameras, and a calibration method for estimating the parameters of the model. The achieved level of calibration accuracy is comparable to the previously reported state-of-the-art.
我们提出了适用于鱼眼镜头相机以及常规和广角镜头相机的通用相机模型，以及用于估计模型参数的校准方法。所达到的校准精度水平与目前的最先进水平相当。

I. Introduction

In this paper, we concentrate on accurate geometric modeling of real cameras. We propose a novel calibration method for fish-eye lenses that requires that the camera observes a planar calibration pattern. The calibration method is based on a generic camera model that will be shown to be suitable for different kind of omnidirectional cameras as well as for conventional cameras. First, in Section II, we present the camera model and, in Section III, theoretically justify it by comparing different projection models. In Section IV, we describe a procedure for estimating the parameters of the camera model, and the experimental results are presented and discussed in Sections V and VI
在本文中，我们专注于相机的精确几何建模。我们提出了一种新的鱼眼镜头校准方法，该方法要求相机观察平面校准图案。校准方法基于通用摄像机模型，该模型将被证明适用于不同类型的全向相机以及常规相机。首先，在第二节中，我们介绍了相机模型，在第三节中，通过比较不同的投影模型从理论上证明了它的合理性。在第四节中，我们描述了估算相机模型参数的程序，实验结果在第五节和第六节中给出和讨论

II. GENERIC CAMERA MODEL

Since the perspective projection model is not suitable for fish-eye lenses we use a more flexible radially symmetric projection model. This basic model is introduced in Section II-A and then extended with asymmetric distortion terms in Section II-B. Computation of back-projections is described in Section II-C.
由于透视投影模型不适合鱼眼透镜，我们使用更灵活的径向对称投影模型。该基本模型在第II-A节中介绍，然后在第II-B节中扩展为不对称失真项。第II-C节描述了反投影的计算过程。

A. Radially Symmetric Model 径向对称模型

The perspective projection of a pinhole camera can be described by the following formula
针孔照相机的透视投影可由以下公式描述
在这里插入图片描述 perspective projection ：透视投影

where θ is the angle between the principal axis and the incoming ray, r is the distance between the image point and the principal point and f is the focal length. Fish-eye lenses instead are usually designed to obey one of the following projections:
其中θ是主轴和入射光线之间的角度，r是像点和主点之间的距离，f是焦距。鱼眼镜片通常设计为符合以下投影之一：
在这里插入图片描述 stereographic projection : 立体投影
equidistance projection : 等距投影
equisolid angle projection : 等角投影
orthogonal projection : 正交投影

Perhaps the most common model is the equidistance projection. The behavior of the different projections is illustrated in Fig. 1(a) and the difference between a pinhole camera and a fish-eye camera is shown in Fig. 1(b)
最常见的模型是等距投影。不同投影的表现如图1（a）所示，针孔相机和鱼眼相机之间的差异如图1（b）所示。
在这里插入图片描述
The real lenses do not, however, exactly follow the designed projection model. From the view-point of automatic calibration, it would also be useful if we had only one model suitable for different types of lenses. Therefore we consider projections in the general form.
然而，真正的镜头并不完全遵循设计的投影模型。从自动校准的角度来看，如果我们只有一个适合不同类型透镜的模型，这也会很有用。因此，我们考虑一般形式的投影
在这里插入图片描述
where, without any loss of generality, even powers have been dropped. This is due to the fact that we may extend r onto the negative side as an odd function while the odd powers span the set of continuous odd functions. For computations we need to fix the number of terms in (6). We found that first five terms, up to the ninth power of θ, give enough degrees of freedom for good approximation of different projection curves. Thus, the radially symmetric part of our camera model contains the five parameters, k1, k2, . . . , k5.
在不失一般性的情况下，甚至连权重都被放弃了。这是因为我们可以将r作为奇函数扩展到负侧，而奇幂跨越连续奇函数集。对于计算，我们需要确定（6）中的项数。我们发现，前五项，直到θ的九次方，给出了足够的自由度，可以很好地逼近不同的投影曲线。因此，我们相机模型的径向对称部分包含五个参数k1、k2、…、k5。

Let F be the mapping from the incoming rays to the normalized image coordinates
设F是从入射光线到归一化图像坐标的映射
在这里插入图片描述 where $r (θ)$ contains the first five terms of (6) and $Φ = (θ, φ)^T$ is the direction of the incoming ray. For real lenses the values of parameters $k_i$ are such that $r (θ)$ is monotonically increasing on the interval $0, θ_{max}]$ , where θmax is the maximum viewing angle. Hence, when computing the inverse of F, we may solve θ by numerically finding the roots of a ninth order polynomial and then choosing the real root between 0 and θmax.
其中 $r (θ)$ 包含（6）的前五项， $Φ = (θ, φ)^T$ 是入射光线的方向。对于真实透镜，参数 $k_i$ 的值使得 $r (θ)$ 在间隔 $0, θ_{max}]$ 上单调增加，其中 $θ_{max}$ 是最大视角。因此，当计算F的逆时，我们可以通过数值计算九阶多项式的根，然后在0和θmax之间选择实根来求解θ。

B. Full Model

Real lenses may deviate from precise radial symmetry and therefore we supplement our model with an asymmetric part. For instance, the lens elements may be inaccurately aligned causing that the projection is not exactly radially symmetric. With conventional lenses this kind of distortion is called decentering distortion [1], [13]. However, there are also other possible sources of imperfections in the optical system and some of them may be difficult to model. For example, the image plane may be tilted with respect to the principal axis or the individual lens elements may not be precisely radially symmetric. Therefore, instead of trying to model all different physical phenomena in the optical system individually, we propose a flexible mathematical distortion model that is just fitted to agree with the observations.
真实透镜可能会偏离精确的径向对称性，因此我们用不对称部分来补充我们的模型。例如，透镜元件可能不准确地对准，导致投影不完全径向对称。对于常规透镜，这种畸变被称为偏心畸变[1]，[13]。然而，光学系统中也存在其他可能的缺陷源，其中一些可能难以建模。例如，图像平面可以相对于主轴倾斜，或者各个透镜元件可以不精确地径向对称。因此，我们不是试图单独模拟光学系统中的所有不同物理现象，而是提出了一个灵活的数学畸变模型，该模型恰好适合于与观测结果一致。

To obtain a widely applicable, flexible model, we propose to use two distortion terms as follows. One distortion term acts in the radial direction
为了获得广泛适用的灵活模型，我们建议使用以下两个畸变项。一个畸变项作用于径向
在这里插入图片描述
and the other in the tangential direction
另一个在切线方向

where the distortion functions are separable in the variables θ and φ. Because the Fourier series of any 2π-periodic continuous function converges in the L2-norm and any continuous odd function can be represented by a series of odd polynomials we could, in principle, model any kind of continuous distortion by simply adding more terms to (8) and (9), as they both now have seven parameters.
其中畸变函数在变量θ和φ中是可分离的。由于任何2π周期连续函数的傅里叶级数收敛于L2范数，并且任何连续奇函数都可以用一系列奇多项式表示，因此原则上，我们可以通过简单地向（8）和（9）添加更多项来模拟任何类型的连续失真，因为它们现在都有七个参数。

七个参数： $l_1,l_2,l_3,i_1,i_2,i_3,i_4$ ;
$m_1,m_2,m_3,j_1,j_2,j_3$ ;

By adding the distortion terms to (7), we obtain the distorted coordinates $x_d = (x_d, y_d)^T$ by
通过将畸变项添加到（7），我们通过以下公式获得畸变坐标 $x_d = (x_d, y_d)^T$

在这里插入图片描述

畸变坐标 = $(8)\mu_r(\psi) + (9)\mu_\psi(\psi)$ = 投影坐标 + 径向畸变项 * 径向单位向量 + 切向畸变项 * 切向单位向量

where $u_r(φ)$ and $u_φ(φ)$ are the unit vectors in the radial and tangential directions. To achieve a complete camera model we still need to transform the sensor plane coordinates into the image pixel coordinates. By assuming that the pixel coordinate system is orthogonal we get the pixel coordinates $u, v)^T$ from
其中 $u_r(φ)$ 和 $u_φ(φ)$ 是径向和切向上的单位向量。为了实现完整的相机模型，我们仍然需要将传感器平面坐标转换为图像像素坐标。通过假设像素坐标系是正交的，我们可以从中获得像素坐标 $u, v)^T$
在这里插入图片描述 where $u_0, v_0)^T$ is the principal point and $m_u$ and $m_v$ give the number of pixels per unit distance in horizontal and vertical directions, respectively.
其中 $u_0, v_0)^T$ 是主点， $m_u$ 和 $m_v$ 分别给出水平和垂直方向上每单位距离的像素数。

By combining (10) and (11) we have the forward camera model
通过组合（10）和（11），我们得到了前向摄像机模型
在这里插入图片描述 where $m = (u, v)^T$ . This full camera model contains 23 parameters and it is denoted by $p_{23}$ in the following. Since the asymmetric part of the model is very flexible, it may sometimes be reasonable to use a reduced camera model in order to avoid over-fitting. This is the case if, for instance, the control points do not cover the whole image area. Leaving out the asymmetric part gives the camera model $p_9$ with nine parameters: five in the radially symmetric part (7) and four in the affine transformation (11). We did experiments also with the six-parametric model $p_6$ which contains only two parameters in the radially symmetric part.
其中 $m = (u, v)^T$ 。该全相机模型包含23个参数，在下面用 $p_{23}$ 表示。由于模型的不对称部分非常灵活，因此有时使用缩小的相机模型以避免过度拟合可能是合理的。例如，如果控制点不覆盖整个图像区域，则会出现这种情况。去掉非对称部分，得到了具有九个参数的相机模型 $p_9$ ：五个在径向对称部分（7），四个在仿射变换（11）。我们还使用六参数模型 $p_6$ 进行了实验，该模型在径向对称部分仅包含两个参数。

径向对称投影（公式6,5个参数）：k1,k2,k3,k4,k5 ;径向畸变（公式8，7个参数）： $l_1,l_2,l_3,i_1,i_2,i_3,i_4$ ;
切向畸变（公式9，7个参数）： $m_1,m_2,m_3,j_1,j_2,j_3$ ; 仿射变换（公式11,4个参数）： $m_u,m_v,u_0,v_0$

C. Backward Model

Above we have described our forward camera model Pc. In practice, one also needs to know the backward model
上面我们已经描述了我们的前向相机模型Pc。在实践中，我们还需要知道后向模型
在这里插入图片描述
which is the mapping from the image point $m = (u, v)^T$ to the direction of an incoming light ray, $Φ = (θ, φ)^T$ . We write $P_c$ as the composite function $P_c = A ◦ D ◦ F$ , where $F$ is the transformation (7) from the ray direction $Φ$ to the ideal Cartesian coordinates $x = (x, y)^T$ on the image plane, $D$ is the distortion mapping from x to the distorted coordinates $x_d = (x_d, y_d)^T$ , and A is the affine transformation (11). We decompose the projection model in this form because, for the inverse transfrom $P^{−1}_c = F^{−1} ◦ D^{−1} ◦ A^{−1}$ , it is straightforward to compute $F^{−1}$ and $A^{−1}$ . The more difficult part is to numerically compute $D^{−1}$ .

这是从图像点 $m = (u, v)^T$ 到入射光线方向 $Φ = (θ, φ)^T$ 的映射。我们将 $P_c$ 写成复合函数 $P_c = A ◦ D ◦ F$ ，其中 $F$ 是从射线方向 $Φ$ 到图像平面上理想笛卡尔坐标 $x = (x, y)^T$ 的变换（7）， $D$ 是从x到畸变坐标 $x_d = (x_d, y_d)^T$ 的畸变映射，A是仿射变换（11）。我们以这种形式分解投影模型，因为对于 $P^{−1}_c = F^{−1} ◦ D^{−1} ◦ A^{−1}$ 的逆变换，计算 $F^{−1}$ 和 $A^{−1}$ 是很简单的。更困难的部分是数值计算 $D^{−1}$ 。

Given a point $x_d$ , finding $x = D^{−1}(x_d)$ is equivalent to computing the shift s into the expression $x = x_d − s$ , where
给定一个点 $x_d$ ，找到 $x= D^{−1}(x_d)$ 相当于将位移s计算为表达式 $x=x_d− s$ ，其中
在这里插入图片描述

位移s = 径向畸变项 * 径向单位向量 + 切向畸变项 * 切向单位向量；同公式（10）的畸变部分

Moreover, we may write $S(Φ) ≡ (S ◦ F^{−1}) (x)$ and approximate the shift by the first order Taylor expansion of $S ◦ F^{−1}$ around $x_d$ that yields
此外，我们可以写 $S(Φ) ≡ (S ◦ F^{−1}) (x)$ ，并通过 $S ◦ F^{−1}$ 在 $x_d$ 周围的一阶泰勒展开来近似转换，从而得出
在这里插入图片描述 where $Φ_d = F^{−1}(x_d)$ may be numerically evaluated. Hence, we may compute the shift s from
其中可以对 $Φ_d = F^{−1}(x_d)$ 进行数值评估。因此，我们可以根据下式计算位移偏移s
where the Jacobians $\partial S / \partial Φ$ and $\partial F / \partial Φ$ may be computed from (14) and (7), respectively. So, finally
其中雅克比 $\partial S / \partial Φ$ 和 $\partial F / \partial Φ$ 可分别由（14）和（7）计算。所以，最后在这里插入图片描述
It seems that the first order approximation for the asymmetric distortion function D is tenable in practice because the backward model error is typically several degrees smaller that the calibration accuracy for the forward model, as will be seen in detail in Section V.
不对称畸变函数D的一阶近似在实践中似乎是成立的，因为后向模型误差通常比前向模型的校准精度小几度，这将在第五节中详细说明。

III. JUSTIFICATION OF THE PROJECTION MODEL 投影模型的正当性

The traditional approach for camera calibration is to take the perspective projection model as a starting point and then supplement it with distortion terms [1], [6], [18]. However, this is not a valid approach for fish-eye lenses because, when θ approaches π/2, the perspective model projects points infinitely far and it is not possible to remove this singularity with the conventional distortion models. Hence, we base our calibration method to the more generic model (6).
相机校准的传统方法是以透视投影模型为起点，然后用失真项[1]、[6]、[18]对其进行补充。然而，这对于鱼眼透镜来说不是一种有效的方法，因为当θ接近π/2时，透视模型将点投射到无限远处，并且不可能用传统的失真模型来消除这种奇异性。因此，我们将校准方法基于更通用的模型（6）。

We compared the polynomial projection model (6) to the two two-parametric models proposed by Miˇcuˇs ́ık [11] for fish-eye lenses. In Fig. 2 we have plotted the projection curves (1)-(5) and their least-squares approximations with models M1, M2 and P3, where P3 is the polynomial model (6) with the first two terms. Here we used the value f = 200 pixels which is a reasonable value for a real camera. The projections were approximated between 0 and θmax, where the values of θmax were 60◦, 110◦, 110◦, 110◦ and 90◦, respectively. The interval [0, θmax] was discretized using the step of 0.1◦ and the models M1 and M2 were fitted by using the Levenberg-Marquardt method. It can be seen from Fig. 2 that the model M1 is not suitable at all for the perspective and stereographic projections and that the model M2 is not accurate for the orthogonal projection.
我们将多项式投影模型（6）与Miácuásık[11]提出的鱼眼透镜的两个双参数模型进行了比较。在图2中，我们用模型M1、M2和P3绘制了投影曲线（1）-（5）及其最小二乘近似，其中P3是具有前两项的多项式模型（6）。这里我们使用了f＝200像素的值，这是真实相机的合理值。投影近似在0和θmax之间，其中θmax的值为60◦, 110◦, 110◦, 110◦ 和90◦, 分别地使用0.1的步长离散区间[0，θmax]◦ 并使用Levenberg-Marquardt方法拟合模型M1和M2。从图2中可以看出，模型M1根本不适用于透视投影和立体投影，模型M2不适用于正交投影。
在这里插入图片描述

In Table I, we have tabulated the maximum approximation errors for each model, i.e., the maximum vertical distances between the desired curve and the approximation in Fig. 2. Here we also have the model P9 which is the polynomial model (6) with the first five terms. It can be seen that the model P3 has the best overall performance from all of the two-parametric models and that the sub-pixel approximation accuracy for all the projection curves requires the five-parametric model P9. These results show that the radially symmetric part of our camera model is well justified.
在表I中，我们列出了每个模型的最大近似误差，即所需曲线与图2中的近似之间的最大垂直距离。这里，我们还有模型P9，它是具有前五项的多项式模型（6）。可以看出，模型P3在所有两个参数模型中具有最佳的总体性能，并且所有投影曲线的亚像素近似精度需要五个参数模型P9。这些结果表明，我们相机模型的径向对称部分是合理的。
在这里插入图片描述

IV. CALIBRATING THE GENERIC MODEL

Next we describe a procedure for estimating the parameters of the camera model. The calibration method is based on viewing a planar object which contains control points in known positions. The advantage over the previous approaches is that also fish-eye lenses, possibly having a field of view larger than 180◦, can be calibrated by simply viewing a planar pattern. In addition, a good accuracy can be achieved if circular control points are used, as described in Section IV-B.
接下来，我们描述用于估计相机模型的参数的过程。该校准方法基于观察在已知位置包含控制点的平面对象。与之前的方法相比，其优势在于鱼眼透镜的视野可能大于180◦, 可以通过简单地观察平面图案来校准。此外，如第IV-B节所述，如果使用圆形控制点，则可以获得良好的精度。

A. Calibration Algorithm

The calibration procedure consists of four steps that are described below. We assume that M control points are observed in N views. For each view, there is a rotation matrix Rj and a translation vector tj describing the position of the camera with respect to the calibration plane such that
校准程序包括以下四个步骤。我们假设在N个视图中观察到M个控制点。对于每个视图，存在描述相机相对于校准平面的位置的旋转矩阵 $R_j$ 和平移向量 $t_j$ ，使得
在这里插入图片描述 We choose the calibration plane to lie in the XY -plane and denote the coordinates of the control point i with $X^i = (X^i, Y^i, 0)^T$ . The corresponding homogeneous coordinates in the calibration plane are denoted by $x^i_p = (X^i, Y^i, 1)^T$ and the observed coordinates in the view j by $m^i_j = (u^i_j , v^i_j )^T$ . The first three steps of the calibration procedure involve only six internal camera parameters and for these we use the short-hand notation $P_6 \hat=(k_1, k_2, m_u, m_v, u_0, v_0)$ . The additional parameters of the full model are inserted only in the final step.
我们选择位于XY平面的校准平面，并用 $X^i = (X^i, Y^i, 0)^T$ .表示控制点 $i$ 的坐标。校准平面中的相应齐次坐标用 $x^i_p = (X^i, Y^i, 1)^T$ 表示，视图 $j$ 中的观察坐标用 $m^i_j = (u^i_j , v^i_j )^T$ .表示。校准程序的前三个步骤仅涉及六个相机内参，对于这些参数，我们使用符号 $P_6 \hat=(k_1, k_2, m_u, m_v, u_0, v_0)$ 表示。完整模型的其他参数仅在最后一个步骤中插入。

Step 1: Initialization of internal parameters

The initial guesses for $k_1$ and $k_2$ are obtained by fitting the model $r = k_1θ + k_2θ^3$ to the desired projection (2)-(4) with the manufacturer’s values for the nominal focal length f and the angle of view $θ_{max}$ . Then we also obtain the radius of the image on the sensor plane by $r_{max} = k_1θ_{max} + k_2θ^3_{max}$ .
$k_1$ 和 $k_2$ 的初始猜测是通过将模型 $r=k_1θ+k_2θ^3$ 与制造商的标称焦距f和视角 $θ_{max}$ 的值拟合到所需投影（2）-（4）来获得的。然后，我们还通过 $r_{max} = k_1θ_{max} + k_2θ^3_{max}$ .获得传感器平面上图像的半径。

With a circular image fish-eye lens, the actual image fills only a circular area inside the image frames. In pixel coordinates, this circle is an ellipse
使用圆形图像鱼眼镜头，实际图像仅填充图像帧内的圆形区域。在像素坐标中，此圆为椭圆
在这里插入图片描述 whose parameters can be estimated. Consequently, we obtain initial guesses for the remaining unknowns $m_u, m_v, u_0,$ and $v_0$ in p, where $m_u = a/r_{max}$ and $m_v = b/r_{max}$ . With a full-frame lens, the best thing is probably to place the principal point to the image center and use the reported values of the pixel dimensions to obtain initial values for $m_u$ and $m_v$ .
其参数可以被估计。因此，我们获得了p中剩余未知数 $m_u、m_v、u_0、$ 和 $v_0$ 的初始猜测，其中 $m_u=a/r_{max}$ ， $m_v=b/r_{max}$ 。对于full-frame 镜头，最好的方法可能是将主点放置到图像中心，并使用reported像素尺寸值获得 $m_u$ 和 $m_v$ 的初始值。

Step 2: Back-projection and computation of homographies

With the internal parameters p6, we may back-project the observed points $m^i_j$ onto the unit sphere centered at the camera origin (see Fig. 1(b)). The points on the sphere are denoted by $\widetilde x^i_j$ . Since the mapping between the points on the calibration plane and on the unit sphere is a central projection, there is a planar homography $H_j$ so that $s\widetilde x^i_j= H_jx^i_p$
使用内参p6，我们可以将观察到的点 $m^i_j$ 反向投影到以相机原点为中心的单位球体上（见图1（b））。球体上的点用 $\widetilde x^i_j$ 表示。由于校准平面上的点和单位球面上的点之间的映射是中心投影，因此存在平面单应性 $H_j$ ，因此 $s\widetilde x^i_j= H_jx^i_p$

For each view $j$ the homography $H_j$ is computed as follows:
对于每个视图 $j$ ，单应性 $H_j$ 计算如下：

(i) Back-project the control points by first computing the normalized image coordinates
通过首先计算归一化图像坐标，反向投影控制点
$\left(\begin{array}{l} x_j^i \\ y_j^i \end{array}\right)=\left[\begin{array}{cc} 1 / m_u & 0 \\ 0 & 1 / m_v \end{array}\right]\left(\begin{array}{l} u_j^i-u_0 \\ v_j^i-v_0 \end{array}\right),$
transforming them to the polar coordinates $\left(r_j^i, \varphi_j^i\right) \hat{=}\left(x_j^i, y_j^i\right)$ , and finally solving $\theta_j^i$ from the cubic equation $k_2\left(\theta_j^i\right)^3+k_1 \theta_j^i-r_j^i=0$ .
将它们转换为极坐标 $\left(r_j^i, \varphi_j^i\right) \hat{=}\left(x_j^i, y_j^i\right)$ ，最后从三次方程 $k_2\left(\theta_j^i\right)^3+k_1 \theta_j^i-r_j^i=0$ 解 $\theta_j^i$

(ii) Set $\tilde{\mathbf{x}}_j^i=\left(\sin \varphi_j^i \sin \theta_j^i, \cos \varphi_j^i \sin \theta_j^i, \cos \theta_j^i\right)^{\top}$ .

(iii) Compute the initial estimate for $\mathbf{H}_j$ from the correspondences $\tilde{\mathbf{x}}_j^i \leftrightarrow \mathbf{x}_{\mathrm{p}}^i$ by the linear algorithm with data normalization [15]. Define $\hat{\mathbf{x}}_j^i$ as the exact image of $\mathbf{x}_{\mathrm{p}}^i$ under $\mathbf{H}_j$ such that $\hat{\mathbf{x}}_j^i=\mathbf{H}_j \mathbf{x}_{\mathrm{p}}^i /\left\|\mathbf{H}_j \mathbf{x}_{\mathrm{p}}^i\right\|$ .
通过具有数据归一化的线性算法，根据对应 $\tilde{\mathbf{x}}_j^i \leftrightarrow \mathbf{x}_{\mathrm{p}}^i$ 计算 $\mathbf{H}_j$ 的初始估计[15]。将 $\hat{\mathbf{x}}_j^i$ 定义为 $\mathbf{H}_j$ 下 $\mathbf{x}_{\mathrm{p}}^i$ 的确切图像，从而使 $\hat{\mathbf{x}}_j^i=\mathbf{H}_j \mathbf{x}_{\mathrm{p}}^i /\left\|\mathbf{H}_j \mathbf{x}_{\mathrm{p}}^i\right\|$

(iv) Refine the homography $\mathbf{H}_j$ by minimizing $\sum_i \sin ^2 \alpha_j^i$ , where $\alpha_j^i$ is the angle between the unit vectors $\tilde{\mathbf{x}}_j^i$ and $\hat{\mathbf{x}}_j^i$
通过最小化 $\sum_i\sin^2\alpha_j^i$ 来优化单应矩阵 $\mathbf H_j$ ，其中 $\alpha_j^ i$ 是单位向量 $\tilde{\mathbf{x}}_j^i$ 和 $\hat{\mathbf{x}}_j^i$ 之间的角度

Step 3: Initialization of external parameters

The initial values for the external camera parameters are extracted from the homographies $H_j$ . It holds that
相机外参的初始值是从单应图 $H_j$ 中提取的
在这里插入图片描述
which implies $H_j = [r^1_j, r^2_j, t_j ]$ , up to scale. Furthermore
这意味着 $H_j=[r^1_j，r^2_j，t_j]$ （按比例）。此外
where $λ_j = sign(H^{3,3}_j )/ ||h^1_j ||$ . Because of estimation errors, the obtained rotation matrices are not orthogonal. Thus, we use the singular value decomposition to compute the closest orthogonal matrices in the sense of Frobenius norm [4] and use them as initial guess for each $R_j$ .
其中 $λ_j = sign(H^{3,3}_j )/ ||h^1_j ||$ 。由于估计误差，所获得的旋转矩阵不正交。因此，我们使用奇异值分解来计算Frobenius范数[4]意义上的最近正交矩阵，并将它们用作每个 $R_j$ 的初始猜测。

Step 4: Minimization of projection error

If the full model $p_{23}$ or the model p9 is used the additional camera parameters are initialized to zero at this stage. As we have the estimates for the internal and external camera parameters, we use (17), (7) or (10), and (11) to compute the imaging function $P_j$ for each camera, where a control point is projected to $\hat m^i_j = P_j (X^i)$ . The camera parameters are refined by minimizing the sum of squared distances between the measured and modeled control point
如果使用完整模型 $p_{23}$ 或模型p9，则在此阶段将附加相机参数初始化为零。由于我们有相机内参和外参的估计，我们使用（17）、（7）或（10）和（11）计算每个相机的成像函数 $P_j$ ，其中控制点投影到 $\hat m^i_j=P_j（X^i）$ 。通过使用LM最小化测量控制点和建模控制点之间的平方距离之和来细化相机参数。
在这里插入图片描述
using the Levenberg–Marquardt algorithm.

B. Modification for Circular Control Points 圆形控制点的修改

In order to achieve an accurate calibration, we used a calibration plane with white circles on black background since the centroids of the projected circles can be detected with a sub-pixel level of accuracy [19]. In this setting, however, the problem is that the centroid of the projected circle is not the image of the center of the original circle. Therefore, since $m^i_j$ in(18) is the measured centroid, we should not project the centers as points $\hat m^i_j$ .
为了实现精确校准，我们使用了黑色背景上带有白色圆圈的校准平面，因为投影圆圈的质心可以以亚像素级别的精度检测[19]。然而，在这种设置中，问题是投影圆的质心不是原始圆中心的图像。因此，由于（18）中的 $m^i_j$ 是测量的质心，因此我们不应将中心投影为点 $\hat m^i_j$ 。

To avoid the problem above, we propose solving the centroids of the projected circles numerically. We parameterize the interior of the circle at $X_0, Y_0)$ with radius R by $X(\varrho, α) = (X_0 + \varrho sin α, Y_0 + \varrho cos α, 0)^T$ . Given the camera parameters, we get the centroid $\hat m$ for the circle by numerically evaluating
为了避免上述问题，我们建议用数值方法求解投影圆的质心。我们通过 $X(\varrho, α) = (X_0 + \varrho sin α, Y_0 + \varrho cos α, 0)^T$ 参数化半径为R的圆 $X_0, Y_0)$ 。给定摄像机参数，我们通过数值计算得到圆的质心在这里插入图片描述 where $\hat m(\varrho, α) = P(X(\varrho, α))$ and $J(\varrho, α)$ is the Jacobian of the composite function $P ◦ X$ . The analytical solving of the Jacobian is rather a tedious task but it can be computed by mathematical software such as Maple.
其中 $\hat m(\varrho, α) = P(X(\varrho, α))$ 和 $J(\varrho, α)$ 是复合函数 $P ◦ X$ 的雅可比。雅可比矩阵的解析解是一项相当乏味的任务，但它可以通过Maple等数学软件来计算。

V. CALIBRATION EXPERIMENTS

VI. CONCLUSION

We have proposed a novel camera calibration method for fish-eye lens cameras that is based on viewing a planar calibration pattern. The experiments verify that the method is easy-to-use and provides a relatively high level of accuracy with circular control points. The proposed camera model is generic, easily expandable and suitable also for conventional cameras with narrow- or wide-angle lenses. The achieved level of accuracy for fish-eye lenses is better than it has been reported with other approaches and, for narrow-angle lenses, it is comparable to the results in [6]. This is promising considering especially the aim of using fish-eye lenses in measurement purposes.
我们提出了一种新的鱼眼镜头相机校准方法，该方法基于查看平面校准图案。实验证明，该方法易于使用，并使用圆形控制点提供了相对较高的精度。所提出的相机模型是通用的，易于扩展，也适用于具有窄或广角镜头的传统相机。鱼眼透镜的精度水平优于其他方法的报道，对于窄角度透镜，其精度与[6]中的结果相当。特别是考虑到在测量中使用鱼眼透镜的目的。

A Generic Camera Model and Calibration Method for Conventional, Wide-Angle, and Fish-Eye Lenses 论文翻译

目录

题目