Homography Matrix, Fundamental Matrix, Essential Matrix
2. Essential matrix
2.1 Essential matrix derivation
2.2 Features
3. Camera internal reference
4. Fundamental matrix fundamental matrix
4.2 Fundamental matrix derivation
4.1 Features
5. Complete model: spatial point to pixel coordinates
6. Homography matrix homography matrix
- Refer to barfoot book p195-
1. Normalize the image coordinates
- Here the camera coordinate system is, the image and the coordinates are O, where the focal length is assumed to be 1; the center of the image coordinates is on the optical axis.
The three-dimensional coordinates of the point P in the space in the camera coordinate system:
Using the triangular approximation relationship, the image coordinates of point P (homogeneous coordinates); at the
same time, assuming that the image coordinates are known, the reverse will find that only two constraint relationships of spatial coordinates can be obtained, or Z can be any value to satisfy the constraint relationship, i.e. depth cannot be obtained;
2. Essential matrix
2.1 Derivation of Essential Matrix
The camera simultaneously observes the spatial point P in two frames of images at different times, as shown in the figure:
Image coordinates of P over two framesSatisfy geometric constraints:
inIt is called the essential matrix, and its parameters are determined by the pose of the motion and have nothing to do with the internal parameters of the camera; the essential matrix is very useful in pose estimation and camera calibration;
Proof of binding relationship:
2.2 Features
- Describe the geometric constraints of a point in space between different frames
- Note that the image coordinates here are the homogeneous coordinates of the projection point of the space point on the camera plane (represented in the camera coordinate system)
- The essential matrix is related to the external parameters of the camera, but not to the internal parameters
3. Camera internal parameters
- Here, it is assumed that the focal length is 1, the center of the image is on the optical axis, the upper left corner of the image is the origin of the pixel coordinates of the image, and the unit of pixel coordinate values is pixels (integer cells);
- The relationship between pixel coordinates and normalized image coordinates (which can be understood as the homogeneous form of projected geometric coordinates of space points on a plane) can be described as:
- K is the camera intrinsic parameter matrix,are the pixel coordinates of the focal length f,是图像中心与图像和光轴交点的偏置（理想情况应该两点重合）；考虑到实际像素感光元是正方形的，参数应该对应近似相等；
4.基本矩阵 fundamental matrix
4.2基本矩阵推导
- 将图像归一化坐标替换为像素点坐标，得到基本矩阵约束（极线约束 epipolar constraint）
- 这里的基本矩阵约束称为极线约束，如下图，如果两帧之间的一个点坐标，外参矩阵已知，则空间点在另外一帧的坐标被约束在一条极线上，可以用来缩小图像匹配点的搜索范围
- 基本矩阵也可以用来极端内参矩阵
4.1特点
- 基本矩阵描述的是不同帧之间同一空间点像素坐标的几何约束关系,由本质矩阵约束中的归一化图像点替换为像素坐标点得到；
- 基本矩阵和相机内参，外参都有关系
- 基本矩阵描述的约束又称为极线约束
5.完整模型：空间点到像素坐标
- 空间点在相机坐标系的坐标到像素坐标的转换关系:
P为齐次坐标到而为坐标的映射矩阵，这个模模型清晰的表示出深度信息的丢失，即无法从像素坐标估计出深度信息。
6.单应矩阵 homography matrix
- 空间点三维坐标到像素坐标（齐次形式）的转换：
- 如果已知观察的点在某个平面上，利用平面方程的先验信息可以估计出深度
- 已知相机中心到平面距离和法向量，可以得平面的法线方程
- 将空间坐标替换为像素坐标：
替换掉,得到由像素齐次坐标到空间坐标的转换
空间点在前后两帧坐标系下的三维坐标的约束关系
带入第一个公式，得到两帧之间同一空间点的像素点坐标的约束关系
将上述公式简写为：
矩阵称为单应矩阵，其中用来度量图像的齐次坐标，可以去掉；所以单应矩阵是pose和平面参数的矩阵；