Camera calibration technology and its applications - monocular camera
First, why should the camera calibration
With the rapid development of machine vision, we have been satisfied with the use of cameras to monitor, capture this relatively simple function. More users favor its use in non-contact measurement of three dimensions. Our so-called three-dimensional measurement of three-dimensional measurement is broad, it includes not only measuring three-dimensional reconstruction of the object, further comprising identifying an arbitrary size and position on the two-dimensional plane in three-dimensional space. This technique has been applied in industrial molds and assembly of high-precision measurements, wherein the size of the detection technology more widely in any two-dimensional plane.
figure 1
1 when the measured plane and the image plane is parallel to and over the imaging model pinhole imaging model, we set the focal length is f, the working distance is d, the measured object and its image OP O'P 'relationship may be a simple Expressed as:
| ON | = | O'P '| × d / f 【1】
But not so in practical applications, we can not strictly control the position of the image plane and the plane of the test, the lens used is not strictly hole model. If the direct use of formula [1] is calculated will have a great error. Accordingly, in order to obtain higher measurement accuracy, we need to achieve conversion and correction coordinates of the image plane by the calibration.
figure 2
Second, what is the camera calibration
In practical applications, uncertainties and lens distortion measured plane so that we can no longer simply use [1] formula to calculate the actual distance, but the data we can currently receiving conversion, these data are consistent with [1 conditions of use] style. That is an arbitrary coordinate plane by rotating and translating over the mapped onto the coordinate plane, the image distortion correction has, let into line with aperture imaging model image plane. With this method, we only determine the conversion algorithm, a correction algorithm and the formula [1] parameters to measure the size and position on any plane in three-dimensional space can be achieved. We call this process of determining parameter called calibration.
Third, the camera unit target set
The method can be divided into single camera calibration target set according to the number of cameras, and multi-objective binocular calibration set. Which is the basis of monocular binocular camera calibration calibration, and calibration of multi-purpose binocular cameras is the camera's expansion. Therefore, we first come today to introduce a single target set. We measure the impact of the photographed image plane deformation of two factors: the attitude of the lens and camera. Based on these two factors will be divided into two groups of parameters of the camera, the camera internal control and external camera parameters.
1. Camera internal reference
image 3
Internal control generally includes a lens of focal length f, lens distortion parameters k, the optical axis of the center coordinates (Cx, Cy) and pixel size Sx, Sy, and when the camera lens is determined, these parameters are uniquely determined. Below we explain in detail the parameters of the mathematical model.
1) focal length
The calculation of different focal length lens and a pinhole model can be divided into telecentric model. 3 we assume that there is any point in the world coordinate system P (x, y), the camera target surface to form the image P '(u, v), the following correspondence between them according to a different optical path model
a) pinhole model
b) telecentric model
Since telecentric lenses special optical design enables the size of the image it has nothing to do with the shooting distance, so the expression is simpler than a pinhole model.
2) lens distortion
Affected by the production and installation of precision lens, we have obtained images will produce nonlinear distortion. We call this distortion lens distortion. Such that the lens distortion errors generated over the pinhole model is no longer applicable. So we need to correct the resulting image first, then apply the ideal pinhole model. Assuming that the original image coordinates (u, v) we acquired, the correction result (u ', v'), the relationship between the coordinates for the distortion model:
a) radial distortion
Figure 4
Radial distortion caused mainly by the surface curvatures of the lens production process, it will make barrel distortion and pincushion distortion (FIG. 4) image generation. The mathematical model is as follows:
Wherein R2 = U2 + V2 , if high accuracy is not required that we can make K2 = 0, K3 = 0 The above equation reduces to the following expression:
b) 离心畸变
离心变量又称偏心变量，它的误差来源于透镜的安装精度，这主要是因为所有镜片的光学中心并不能严格的保证在同一条直线上。这种误差除了在引入径向畸变同时还会引入切向畸变。由于之前我们已经进行了径向畸变的校正，因此我们在此基础上只需加入切向畸变校正即可。其数学模型如下：
c) 薄棱镜畸变
影响薄棱镜畸变的主要因素的是透镜以及摄像机靶面的平行度，镜片与摄像机靶面夹角越大畸变就越严重。其数学表达式如下：
d) 畸变校正
在实际的应用中，大多数工业摄像机的厂商可以通过摄像机接口螺纹的机械精度来保证镜头透镜与靶面的平行性，而且这种畸变产生的误差较小，因此在一般的图像标定中不作考虑。至此，我们已经基本掌握了大多数情况下畸变产生的原因以及数学模型。结合【5】、【7】两式我们可以推导出镜头畸变校正模型：
2. 摄像机外参
摄像机的外参是指摄像机坐标系与世界坐标系的转换参数它主要由旋转矩阵R和平移矩阵T组成。对于任意三维坐标系，我们都可以通过这两个矩阵将其转换到摄像机坐标系中。其数学模型为c=(x,y,z)T=R∙Pw+T 【10】
1) 旋转矩阵R
如图5所示，我们通过沿坐标轴x、y、z分别旋转α、β、γ来实现坐标系的转换。因此，旋转矩阵R可分解为Rx（α）、Ry（β）、Rz（γ）三个矩阵相乘的形式。我们以z轴为例，假设(x0,y0)与x轴夹角为θ，且到原点距离为r，通过旋转矩阵
Rz（γ）坐标系沿z轴旋转γ后得到点(x1,y1)，我们可得方程组:
x1=r·cos(θ+γ) 【11】
y1=r·sin(θ+γ) 【12】
由三角函数展开得:
x1= r·cos(θ) cos(γ)- r·sin(θ) sin(γ) 【13】
y1= r·sin(θ) cos(γ)+ r·cos(θ) sin(γ) 【14】
由(x0,y0)与x轴夹角为θ得：
x0=r·cos(θ) 【15】
y0=r·sin(θ) 【16】
将【15】式带入【13】式、【16】式带入【14】式得：
x1=x0·cos(γ)-y0·sin(γ) 【17】
y1=y0·cos(γ)+x0·sin(γ) 【18】
因此
以此类推求得Rx（α）和Ry（β）将它们与Rz（γ）相乘得：
【20】
2) 平移矩阵T
通过旋转矩阵运算后，世界坐标系的三个坐标轴会与摄像机坐标系对应的坐标轴相平行。此时我们已经离我们的目标又近了一步。如图7所示我们现在只要沿各坐标轴做平移运算即可，由此得：
T=(tx,ty,tz)T 【21】
3) 参数求解
根据【2】、【9】、【20】、【21】这几个数学模型，我们可以得知，若想确定一个摄像机与被测平面的相对位置，则需要确定包括内参、外参在内的14个参数(f,κ1,κ2,κ3,sx,sy,cx,cy,α,β,γ,tx,ty,tz) ，其中f,sx,sy,cx,cy是已知的。因此，我们至少需要9个坐标点，构成9个方程才可以解出剩余的9个未知数。在通常情况下，点的分布以覆盖大部分视场为准，获得的数据点越多，统计的参数就越准确。我们一般采用最小二乘法或者线性规划等统计算法来求解相应参数。下面我们以HALCON为例演示一个标定的全过程。
HALCON是德国MVTEC Software GmbH公司开发的一套完善的机器视觉算法软件包。它除了拥有亚像素精度的算法以及高效的处理性能外，在三维重构方面它也有卓越的表现。它的开发环境中自带摄像机标定工具，可以轻松的完成摄像机的标定工作。同时，您还可以使用HALCON生成可打印的标定板文件。下面我们就来介绍一下摄像机标定的整个流程。
a) 生成标定板
i. 创建标定板
使用HALCON开发环境HDevelop创建标定板
在选择尺寸选择时推荐大家使用边长为视野1/3左右的标定板
ii. 打印标定板
通过GSView等高精度打印软件打印标定板。
b) 摄像机参数设置
输入所使用的摄像机以及镜头的相关参数
c) 拍摄标定板图像
我们采用平移和倾斜的方式使得拍摄图像中的标定板尽量覆盖整个视场。
一般情况我们需要保存15幅不同位置的图像，具体位置如下：
d) 标定图像载入
通过HALCON我们可以实时拍摄图像也可以读取我们事先拍摄好的图像进行标定。如果标定板识别成功，图像上将绘制出标定板坐标系。
e) 标定
标定后我们会获得摄像机内参和外参。我们还可以将它们保存起来用于坐标转换或图像校正。
f) 验证标定结果
在HALCON中不仅有摄像机标定工具还有测量工具，下面我们使用HDevelop自带的一维测量助手来验证一下我们的标定结果。
首先我们要加载我们刚才标定的数据。
成功加载标定数据后我们就可以使用我们随机拍摄的一张图像进行测量。
通过以上几步操作，现在我们就已经得到了像素点的实际距离。
图14
Complex calibration work with the help of HALCON would easily done. Not only that, all of the above steps can be exported to the code, we can integrate this into our own program.
NOTE: If high-precision calibration plates will be more accurate calibration results.
Fourth, the application:
Monocular camera calibration technique is applied to the measured surface curvature smaller and need to obtain the actual data applications, such as food, machinery and semiconductors. Theoretically, all of the detection process determines the size of the analyte can be used to complete the pixel size. While the pixel size and the actual size of the computer is almost the same, generally used only for size comparison, but for us, the actual size is more intuitive.
15 is an example of an image processing detection system frozen fish fillets. Depending on the practical application of fillet width different batches, therefore, the best way is to use the calibration to obtain the actual data is calculated. This makes the system parameter determining method more intuitive, easier to set. FIG 16 is a workpiece size detection system by calibration, we can obtain not only the actual data, but they can be directly compared CAD data, improve the detection efficiency.
V. Conclusion:
In the machine vision industry booming today, the camera calibration has been gradually applied to medical care, food, abrasive production, semiconductor manufacturing, and many other detection systems. With the continuous expansion algorithm library HALCON user base like this, as the camera calibration in our research, it would also continue to walk into industrial applications. It will provide better, more accurate solutions of two-dimensional and three-dimensional space for us to be the driving force increase production efficiency and product quality.
