kinectv1 external parameter calibration (Opencv's epnp)
illustrate
1. kinectv1 external parameter calibration
2. Use Opencv's epnp function to realize
3. Python programming
1. Calibration requirements
The pixel coordinates (u, v) are obtained according to the color camera of kinectv1, and the Zc corresponding to (u, v) in the camera coordinate system is obtained through the depth camera of kinectv1, and then the real world coordinates of the point are restored according to the obtained internal and external parameters of the camera.
If we do not have depth information, we can only calculate the world coordinates of the plane, and the actual Z value cannot be restored. Read this blog , which includes the principle of camera calibration and the relationship and practical significance of each coordinate system involved in calibration.
2. Camera Introduction
The actual meaning of depth information is the vertical distance from the point to the camera optical axis plane, that is, the Zc value in the camera coordinate system
3. Internal reference calibration
Calibration of kinect camera's internal parameters
can be done with color calibration files in actual use
4. Calibration method
1. Principles and practical related articles
Use the pnp method for calibration. There are many pnp calibration principles on the Internet. I think the principle of this article is quite good, and I think the application of this article is very good.
Next catalog:
"Camera Pose Estimation 0: Basic Principles How to Solve PNP Problems"
"Camera Pose Estimation 1: Estimating Camera Pose Based on Four Feature Points"
"Camera Pose Estimation 1_1: OpenCV:solvePnP Secondary Packaging and Performance Test"
"Camera Pose Estimation 2: [Application] Real-time Pose Estimation and 3D Reconstruction Camera Pose"
"Camera Pose Estimation 3: Finding the World Coordinates of a Point Based on the Pose Estimation Results of Two Images"
Why does pnp solve Need 4 pairs of coordinates: the most popular geometric image interpretation method
2. The pnp calibration function solvepnp in opencv
retval1, rvec1, tvec1=cv.solvePnP(world_point,pixel_point,K,distortion_coefficients)
The parameters of this function are input in order of world coordinates, pixel coordinates, internal reference matrix, distortion matrix, and the number of iterations. The
last parameter of the method has the following three types. The blogger uses iterative solution and epnp. Both methods can restore the world coordinates. The difference not big.
5. Calibration experiment
The height of the camera is 1m above the tabletop, the optical axis plane and the tabletop have a certain angle of inclination, and each grid of the checkerboard is 4cm.
1. Point selection
When using the default method, select the four corners in the figure below as calibration points.
When using epnp calibration, select
the light blue point in the figure below for the first control point,
select the blue point for the second control point,
select the red point for the third control point, and
select the green point for the fourth control point
The direction of the world coordinates,
the vertical desktop upwards is the positive direction of the Z axis,
the blue point to the red point is the negative direction of the Y axis,
and the blue point to the green point is the positive direction of the X axis
2. epnp function call code (python)
# encoding: utf-8
import cv2 as cv
import numpy as np
import yaml
import os
import io
with open("file/rgb_A22593W01131223A.yaml", 'rb') as f: #写入正确的文件路径
data = yaml.load(f)
camera_matrix = data['camera_matrix']['data']
distortion_coefficients = data['distortion_coefficients']['data']
distortion_coefficients=np.array(distortion_coefficients)
K= np.zeros([3, 3], dtype=float)
# K=np.mat(K)
K[0,0]=camera_matrix[0]
K[0,1]=camera_matrix[1]
K[0,2]=camera_matrix[2]
K[1,0]=camera_matrix[3]
K[1,1]=camera_matrix[4]
K[1,2]=camera_matrix[5]
#
K[2,0]=camera_matrix[6]
K[2,1]=camera_matrix[7]
K[2,2]=camera_matrix[8]
pixel_point= np.array([
[328.0, 172.0],
[158.0, 58.0],
[137.0, 310.0],
[508.0, 55.0]
])
world_point= np.array([
[360.0, -280.0, 90.0],
[0.0, 0.0, 0.0],
[0.0, -520.0, 0.0],
[720.0, 0.0, 0.0]
])
retval1, rvec1, tvec1=cv.solvePnP(world_point,pixel_point,K,distortion_coefficients, useExtrinsicGuess=1, flags=cv.SOLVEPNP_EPNP)
r1 = cv.Rodrigues(rvec1,jacobian=0)
print(r1[0])
print(tvec1)
3. Analysis of results
(1) Pixel coordinates and actual coordinates
pixel_point= np.array([
[328.0, 172.0],
[158.0, 58.0],
[137.0, 310.0],
[508.0, 55.0]
])
world_point= np.array([
[360.0, -280.0, 90.0],
[0.0, 0.0, 0.0],
[0.0, -520.0, 0.0],
[720.0, 0.0, 0.0]
])
(2) External parameter matrix
[[ 9.98539408e-01 1.49732959e-02 -5.19119596e-02 -3.28108974e+02]
[ 3.34250968e-03 -9.76109922e-01 -2.17251577e-01 -3.78811896e+02]
[-5.39247510e-02 2.16760745e-01 -9.74734272e-01 1.18536981e+03]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00 1.00000000e+00]]
(3) Calculation results
For the verification of four control points and two arbitrary points, the depth value obtained by the camera itself is compensated by 7 cm.
coordinates of two arbitrary points
720,-520, 0
200,-80,25
result
[[ 3.59876941e+02 -5.67379331e-01 -7.67101745e+00 7.19528649e+02 7.25095153e+02 2.00069031e+02]
[-2.80541796e+02 7.66719036e-01 -5.24430480e+02 -7.01423857e-01 -5.22356724e+02 -8.39731426e+01]
[ 9.35158355e+01 1.39182581e+01 -6.04520752e+00 8.63555196e+00 -9.18937150e+00 2.29591386e+01]
[ 1.00000000e+00 1.00000000e+00 1.00000000e+00 1.00000000e+00 1.00000000e+00 1.00000000e+00]]
The error is within one centimeter