Unity Shader graphics, perspective projection matrix, the principle and derivation of orthogonal projection matrix, the most intuitive understanding of the meaning of the matrix

Preface

When I was reading "Introduction to UnityShader", I found that the book did not give the derivation process of the projection matrix
The projection matrix plays a key role in graphics and in games Development, virtual reality and augmented reality, robotics, and machine vision are all involved
I want to explain how perspective orthogonal projection is obtained in the most intuitive way
If you want to understand this article, you must have a clear and thorough understanding of zoom, rotation, and translation, and the three major matrices

Orthographic projection

Let’s start with a simple orthogonal projection. If you want to learn the perspective matrix, it is recommended to learn the orthogonal matrix first

Size: half the height in the vertical direction, NearHeight/2
Aspect, camera aspect ratio, Aspect=$\frac{NearWidth}{NearHeight}$
Find the transformation matrix based on the known Size, Near, Far, and Apect

First scale the cuboid so that xyz is in [-1,1],
As shown in the figure below,
for the y-axis, for point 2.y=$\frac{NearHeight}{2}$=Size,Size=>1,k₂=$\frac{1}{Sifrom}$
The intuitive understanding is that the y-axis is from Size=>1, and the scaling size of the y-axis is$\frac{1}{Sifrom}$
Aspect(简称a)=$\frac{NearWidth}{NearHeight}$,w=ah,x=ay

对于x轴
Attention is change before a=x/y => x=ay
x'=k< a i=3>1x y'=k₂y₁ y x \frac{y}{x} < /span> * ₂=k₁ ky₂x=k k x'=y'=1(变换之后的x',y'弐标都为1)



$\frac{y}{x}$
a=$\frac{x}{and}$
k1=$\frac{k2}{a}$
k2=$\frac{1}{Sifrom}$
k1=$\frac{1}{Aspect\ast Size}$
w=ahWhy not k₁=ak₂, but k₁=k₂/a, which is Because what we know is the ratio before transformation, after transformation x'=y',
Assume it is a horizontal screen, the width is greater than the height, that is, a>1, x>y, we want x'=y ', then x'=kx,k<1, that is, 1/a

对于z轴,观观查长方体变换前后的长率,z=Far-Near,z'=2,z轴下载、乘い-1,
Far-Near= >2
z'=k₃z,
k − 2 F a r − N e a r -\frac{2}{Far-Near} =₃$-\frac{2}{Far-Near}$
矩阵:
$$\left[\begin{array}{cccc}\frac{1}{Sifrom}& 0& 0& 0\\ 0& \frac{1}{AspectSize}& 0& 0\\ 0& 0& -\frac{2}{Far-Near}& ?\\ 0& 0& 0& 1\end{array}\right]$$
Then consider the translation on the z-axis to find the complete matrix
The translation consists of 2 parts, as shown in the figure above, in the zoomed space observation, < /span>
One part is the distance from the camera to the clipping plane, and the other part is half the length of the cube.
The distance from the camera to the clipping plane. Due to scaling, the size is not Near, but Near*k₃=< a i=3> − 2 N e a r F a r − N e a r -\frac{2Near}{Far-Near} $-\frac{2Near}{Far-Near}$
Half the length of the cube, note that the z-axis is flipped, 1,$-\frac{2Near}{Far-Near}$-1=$-\frac{2Near}{Far-Near}$-$\frac{Far-Near}{Far-Near}$
=$\frac{-Near-Far}{Far-Near}$=$-\frac{Far+Near}{Far-Near}$(Consistent with the writing method in the book)
At this point, the orthogonal matrix has been derived
$$\left[\begin{array}{cccc}\frac{1}{Sifrom}& 0& 0& 0\\ 0& \frac{1}{AspectSize}& 0& 0\\ 0& 0& -\frac{2}{Far-Near}& -\frac{Far+Near}{Far-Near}\\ 0& 0& 0& 1\end{array}\right]$$
If you don’t understand why matrices are written, you can look up information about scaling, rotation, and translation, and the three major matrices.

perspective projection

1. Basic concepts:
   NearHeight: the height of the near clipping plane
   FarHeight: the height of the far clipping plane
   Near :The distance of the camera from the near clipping plane
   Far:The distance of the camera from the near clipping plane
   FOV:Field of View,field of view

Why can’t the above transformation be directly transformed into a [-1,1] standardized cube like an orthogonal transformation?
Instead, it must be transformed into a prism. This is because the matrix Linear transformation can transform a cuboid into a cube,
cannot transform a prism into a cube, and converting a prism into a cube requires a nonlinear transformation function,
4 The transformation matrix of The straight lines on both sides of the prism are not parallel => parallel, linear transformation cannot be achieved
Store z before transformation in w, divide all points xyz after transformation by w, and then transform them to [-1,1], this transformation is non-linear, and the scaling value is not fixed. For
different planes (planes parallel to xOy), the scaling value is the distance from the camera before the plane
This is why xy has Near,Far after transformation
The above transformation can be divided into the following steps

• Scale the view frustum so that its xyzw is equal to the scaled xyzw
• Translate the viewing frustum in the z direction
• Flip z-axis

tan$\frac{FOV}{2}$=$\frac{\frac{nearHeight}{2}}{Near}$ ①
tan$\frac{FOV}{2}$Equal to half the height of the near clipping plane/Near
Aspect, camera aspect ratio, Aspect=$\frac{NearWidth}{NearHeight}$
First consider scaling
The scaling of the y-axis, for the following figure point 2.y< /span>$\frac{NearHeight}{2}$=>Near
对于y轴,由$\frac{nearHeight}{2}=>Near$,
Sokufugenori y 乘い$\frac{Near}{\frac{NearHeight}{2}}$=>y’
y*$\frac{Near}{\frac{NearHeight}{2}}$=y’
y’=ky,k=$\frac{Near}{\frac{NearHeight}{2}}$②
由①=>cot$\frac{FOV}{2}$=$\frac{Near}{\frac{nearHeight}{2}}$③
由②③gotk=cot$\frac{FOV}{2}$
How to intuitively understand cot$\frac{FOV}{2}$,
is actually the scaling of point 2.y before and after the y transformation,
Exactly equal to half the height of Near/near clipping plane, that is, cot$\frac{FOV}{2}$
因为,NearWidth=Aspect*NearHeight,w=ah
NearHeight=$\frac{NearWidth}{Aspect}$,h=w/a
For x-axis scaling=y-axis scaling/Aspect
For x-axis, by NearWidth/2= >Near, that is, multiply the original x by$\frac{Near}{\frac{NearWidth}{2}}$
w=ahWhy not k₁=ak₂, but k₁=k₂/a, which is Because what we know is the ratio before transformation, after transformation x'=y',
Assume it is a horizontal screen, the width is greater than the height, that is, a>1, x>y, we want x'=y ', then x'=kx,k<1, that is, 1/a
There are two ways to derive the scaling of the x-axis
Method 1:
Be careful before changing a=x/y => x=ay
x'=k₁x
y'=k₂y
x'=y' =Near(变换之后的x',y'弐标都为Near)
k₁x=k=k< /span> y x \frac{y}{x} * ₂₁ ky₂
$\frac{y}{x}$
a=$\frac{x}{and}$
k1=$\frac{k2}{a}$
k2=cot$\frac{FOV}{2}$
k1=$\frac{whatt\frac{FOV}{2}}{Aspect}$
方法2:
x * $\frac{Near}{\frac{NearWidth}{2}}$=x’,k2=$\frac{Near}{\frac{NearWidth}{2}}$
②=>$\frac{Near}{\frac{h}{2}}$=>$\frac{Near}{\frac{w/a}{2}}$=>$\frac{Near}{\frac{w}{2a}}$
cot$\frac{FOV}{2}$=$\frac{Near}{\frac{nearHeight}{2}}$③
=>cot$\frac{FOV}{2}$=$\frac{Near}{\frac{w}{2a}}$
=>$\frac{whatt\frac{FOV}{2}}{a}$=$\frac{Near}{\frac{w}{2}}$
k2=$\frac{Near}{\frac{w}{2}}$=$\frac{whatt\frac{FOV}{2}}{a}$
Therefore, the scaling factor of x, y has been obtained
$$\left[\begin{array}{cccc}\frac{whatt\frac{FOV}{2}}{Aspect}& 0& 0& 0\\ 0& whatt\frac{FOV}{2}& 0& 0\\ 0& 0& ?& ?\\ 0& 0& ?& 0\end{array}\right]$$
Same as the picture above, easy to view the picture

The following explains how to obtain the scaling of the z-axis
For the view frustum, in the z direction, the length of the previous view frustum is Far-Near, and the size after transformation is Far+Near ,
So k₃=$\frac{Far+Near}{Far-Near}$,对z轴进行訳,k₃=-$\frac{Far+Near}{Far-Near}$
The following explains the translation of the z-axis and the flip of the z-axis. The translation is divided into two parts. Observed in the zoomed space,
The part is the z of the transformed point 2, that is, z'₂ =k_{3< /span>}Near, the other part is Near
z’=k₃z,k₃=$\frac{Far+Near}{Near-Far}$,Near$\frac{Far+Near}{Near-Far}$=$\frac{Near\ast Far+Nea{r}^2}{Near-Far}$
After adding the transformation, -Near (because of the flip of the z-axis),$\frac{Near\ast Far+Nea{r}^2}{Near-Far}$-Near=
$\frac{Near\ast Far+Nea{r}^2}{Near-Far}$-$\frac{Nea{r}^2-Near\ast Far}{Near-Far}$= $\frac{2\ast Near\ast Far}{Near-Far}$
At this point, the zoom and pan are completed
To save the previous z, just set the 4th row and 3rd column to 1
The perspective matrix is Derived
$$\left[\begin{array}{cccc}\frac{whatt\frac{FOV}{2}}{Aspect}& 0& 0& 0\\ 0& whatt\frac{FOV}{2}& 0& 0\\ 0& 0& -\frac{Far+Near}{Far-Near}& -\frac{2\ast Near\ast Far}{Far-Near}\\ 0& 0& 1& 0\end{array}\right]$$