Hello everyone, I am Zhao.
Many people on the Internet have introduced the rendering pipeline. This is basically a note written for myself. Don’t mind.
1. A brief description of the rendering pipeline:
If the rendering pipeline is listed, there are probably these processes:
CPU model data - vertex control program - tessellation - geometry shader - clipping - NDC space - screen space - raster - fragment handler - frame buffer - postprocessing Among them
:
1. In the stage of CPU processing model data, the general game engine parses in a fixed format, which does not require manual processing by the user, but only requires the user to assemble the data according to the given API.
2. Tessellation and geometry shaders are optional and generally not available.
3. The process from clipping to rasterization is generally low-level processing, and we basically don’t need to operate it.
4. Therefore, for those who use off-the-shelf 3D engines, the only processes that can interfere are the vertex control program and the fragment processing program. . We can achieve these two processes by writing shader. The specific thing we can do is to change the shape and color of the object.
Second, the specific implementation of each step:
If you just use the 3D engine, in fact, the following process can be completely ignored. However, based on understanding the principles, it will also be helpful at the usage level, or when one day you need to write your own engine, you still need to understand the knowledge of these processes. Ah Zhao, I tried to write my own engine almost ten years ago. Due to my limited ability, the engine I wrote can only reach the level of rendering 3D models and playing skeletal animations, but the whole process should still be complete.
Let's talk about it step by step:
1. CPU stage
1. Object removal
Purpose: Eliminate things that the camera should not see in the scene, and reduce the number of rendered objects
(1) Frustum culling (cone and object for collision detection)
(2) Layer (Layer) culling
(3) Occlusion culling
step All pure CPU things, before being pushed into the GPU for rendering, some objects are filtered out through certain conditions to reduce the rendering pressure of the GPU. But everything is relative. After completing these steps, the GPU pressure is reduced, but it will consume CPU performance. So I feel that when optimizing, you should control it according to your own situation. If you don’t do it well, it may become a negative optimization.
If you can use Layer to cull, this should be the least expensive.
Then the culling of the viewing cone is simply a vertex collision detection. First exclude all objects whose vertices are not within the frustum.
The last is occlusion culling. I personally feel that this is a means of exchanging CPU and memory performance for GPU performance. For example, Unity's occlusion removal function needs to mark the object as static first, and then bake it to generate occlusion data, so as to hide the unseen objects according to the range seen by the camera.
2. Rendering sort
Purpose: To determine the order of object rendering
The rendering of translucent objects has always been a problem worthy of research in the 3D engine. Different rendering rules can be specified through the rendering order.
Rules of the Unity engine
(1) Rendering order <2500 is opaque object
sorting order: according to the distance from the camera: from front to back sorting
(2) rendering order >=2500 is translucent object
sorting order: according to the distance from the camera: from back to front
3. Submit information to GPU
Submit through the command: SetPassCall, DrawCall The submitted
information includes:
(1) Model information
(2) Transformation matrix
(3) Light source information
(4) Shader used
(5) Parameters on the material ball
At this step, the work of the CPU is That's over, the rest is the GPU's job.
2. Vertex processing
Function: change the shape of an object
1. Convert from object local coordinates (Local) to world coordinates (World)
2. Convert from world coordinates to camera coordinates (View)
3. Conversion from camera coordinates to clipping space coordinates (Clip)
The simple implementation of this process is to multiply the MVP matrix and vertexPos, and finally convert the vertex coordinates of the model from local coordinates to clipping space coordinates.
However, when actually writing the shader, we don't necessarily want to get the clipping space coordinates of a vertex in a regular manner. We can also do many things, such as obtaining data such as normals and tangents, such as processing vertex colors, distorting vertex positions, etc. We can add fields to the custom data structure and calculate the results in the vertex program. The result is then saved for use by the fragment shader.
3. Rasterization operation stage
This stage is just a general term. In fact, before the rasterization operation, it is necessary to process the model information and convert their coordinate system, homogeneous clipping space (CVV) - normalized device coordinate space (NDC) - screen coordinates ( Screen), we can get what is actually displayed on the screen
(1) cropping
Cut off the edges of the triangles that intersect with the frustum clipping space, and generate new triangles.
In the clipping space, it is relatively easy to judge whether the coordinates of the point exceed the clipping range (cvv space). The step of generating new triangles is theoretically necessary, but from the perspective of actual rendering, whether it is faster to regenerate triangles according to the clipping edge, or to render all triangles as long as there is one vertex within the clipping range, this I also don’t know. Not sure.
(2) Convert from CVV space to NDC
NDC space (normalized device coordinate space), which is a space between the clipping space and the screen coordinate space, is obtained by dividing the coordinate xyz of the clipping space by the w component, and the coordinate ranges of its three axes are all Yes is -1 to 1.
(3) Backface culling
Purpose: If backface culling is turned on, it is necessary to cull the triangular face part on the backside so that it does not participate in rendering.
Judgment method:
Judging by the relationship of the three vertices of the triangle in the NDC coordinates: clockwise is the back, counterclockwise is the front. Triangles that belong to the back (back to the camera) can be removed
(4) Convert NDC coordinates to screen coordinates
Convert the coordinates of the NDC value range from -1 to 1 to the actual coordinates on the screen.
Calculation method:
ScreenX = NDC.x * pixelWidth/2 + pixelWidth/2;
ScreenY = NDC.y * pixelWidth/2 + pixelWidth/2;
(5) Graphic element assembly
The previous steps operate on isolated vertices. From this step, the points are indexed and assembled into triangular graphics (primitives)
(6) Rasterization
Decomposing the previous triangle graphics according to a certain unit (which can be understood as a pixel), it can be considered that each fragment (fragment) obtained is a pixel. So it can be generally understood that in the fragment shader, we change the color of a certain pixel.
4. Fragment processing
Function: change the pixel color
1. Light Shading
Changing the color through the lighting model
I have previously shared an article about the lighting model. The general lighting model consists of:
ambient light + diffuse reflection color + highlight color
2. Texture Shading
Use UV coordinates to sample the texture map and get the corresponding color
5. Output merge
Function: Decide whether the current pixel should output something.
1. Alpha test
2. Stencil test
3. Depth test
4. Color mixing
6. Frame buffer
The display output content of a certain frame is saved in the frame buffer, and can be processed again after being obtained.
7. Post-processing
Perform post-processing on the saved frame buffer content