[Computer Graphics] [GAMES101 Study Notes] Rasterization Rasterization

1. Rasterization overview

Rasterization is the central operation of object sequential graphics, and the rasterizer is the center of any graphics pipeline. For each primitive input, the rasterizer has two tasks: enumerate the pixels covered by the primitive, and interpolate values, called attributes, over the primitive. The purpose of these properties will be made clear in the examples that follow. The output of the rasterizer is a set of fragments, one for each pixel covered by the primitive. Each fragment "exists" at a particular pixel and carries its own set of attribute values.

The popular understanding is that the MVP operation mentioned above first presents the object in $[-1,1{]}^3$ standard space, and then mapped to [ 0 , width ] ∗ [ 0 , height ] [0, width] * [0, height]after viewport transformation$[0,width]\ast [0,height]$ screen, and finally through rasterization to actually display the object on the screen. In other words, the purpose of rasterization is to draw objects on the screen.

2. Understanding pixels and screens

A pixel refers to the smallest unit in an image represented by a sequence of numbers. Usually, an image can be regarded as a two-dimensional matrix. For example, an image of 800 x 800 pixels is actually a matrix of 800 x 800. Each matrix in the matrix An element is actually a pixel.

The clever thing is that in computer graphics, the screen is also a two-dimensional array of pixels, and what we usually call resolution (such as 1920*1080/1080p) is the size of the pixel array. Generally speaking, the more pixels, the larger the pixel array, and the higher the screen resolution. The screen is a typical raster display. The details are as follows:

• Each pixel is presented in the form of a square, and the coordinate range is $(0,0)$到$(width-1,height-1)$；
• The coordinates of the pixel center point are the pixel coordinates, and the value is expressed as $(x+0.5,y+0.5)$；
• The entire screen represents the range $(0,0)$到$(width,height)$。

3. Linear Rasterization

3.1 DDA numerical differentiation algorithm

The DDA (Differential Numerical Algorithm) algorithm is an incremental algorithm. Incremental algorithm: In an iterative algorithm, the x and y values ​​at each step are obtained by adding an increment to the value at the previous step. Construct a set of virtual gridlines through the centers of pixels in each row and column. Calculate the intersection point of the line and each vertical grid line in the order of the line from the start point to the end point, and then determine the pixel closest to the intersection point in the column of pixels according to the sign of the error term.
DDA needs to consider the slope k of the line drawn:

• 当$|k|<1$，$x$ increases by 1,$y$ increases$k$；
• When $|k|>1$，yyFor every 1 increase$in\; y$$x$ increases by$1/k$。

void CMyView::OnDdaline()
{
    
    
    CDC *pDC=GetDC();              // 获得设备指针
    intx0=100,y0=100,x1=300,y1=200,c=RGB(255,0,0);  //定义直线两端点和直线颜色（红色）
    float x,y,i;
    float dx,dy,k;
    dx=(float)(x1-x0);
    dy=(float)(y1-y0);
    k=dy/dx;//计算斜率
    y=y0; x=x0;
    if(abs(k)<1)
    {
    
       for(;x<=x1;x++)
        {
    
    pDC->SetPixel(x,int(y+0.5),c);
        y=y+k;}//x自增，y=y+k
    }
    if(abs(k)>=1)
    {
    
    
      for(;y<=y1;y++)
      {
    
    pDC->SetPixel(int(x+0.5),y,c);
      x=x+1/k;}
    }
    ReleaseDC(pDC);      //释放设备指针
}

3.2 Midpoint Bresenham Algorithm

The Bresenham algorithm is the most widely used line scan conversion method in the field of computer graphics. The principle is: construct a set of virtual grid lines through the pixel centers of each row and column, calculate the intersection point of each vertical grid line of the line in the order of the line from the starting point to the end point, and then determine the pixel closest to the intersection point in the column of pixels. The advantage of this algorithm is that incremental calculations can be used, so that for each column, the pixel sought for that column can be determined by simply checking the sign of one error term.

void CTestView::OnBresenhamline()
{
    
    
    CDC*pDC=GetDC();
    intx1=100,y1=200,x2=600,y2=800,color=RGB(0,0,255);
    inti,x,y,dx,dy;
    float k,e;
    dx=x2-x1;
    dy=y2-y1;
    k=dy/dx;
    e=-0.5;  x=x1; y=y1;//e初值d0-0.5
    for(i=0;i<=dx;i++)
    {
    
       pDC->SetPixel(x,y,color);
       x++;
       e=e+k;
       if(e>=0)  {
    
     y++; e=e-1;}
    }
}

4. Triangle Rasterization

4.1 Benefits of triangles

• Is the most basic polygon, any polygon can be split into triangles;
• It can be guaranteed that the three vertices are on the same plane;
• The inside and outside of the triangle are well-defined, which facilitates the shading of pixels;
• Well-defined method for interpolation at triangle vertices (barycentric interpolation).

4.2 How to rasterize

To realize the rasterization of the triangle, it is necessary to sample the pixels, that is, to traverse all the pixels to determine whether the pixels are in the triangle. The corresponding code is as follows:

for(int x = 0 ; x < xmax ; ++x){
    
    
	for(int y = 0 ; y < ymax ; ++y){
    
       //遍历每个像素点
		image[x][y] = inside(tri,x+0.5,y+0.5);   //在三角形内部的像素点作为采样对象
	}
}

The inside function is a function used to determine whether a pixel is inside a triangle. The most classic implementation method is to use cross multiplication, as shown in the figure below:

We start from $P_2$In clockwise order, the straight line $P_2{P}_1$and straight line $P_{2}The\; cross\; product\; of\; Q$ , use the right-hand rule, point to the screen, point$Q$ in line$P_2{P}_1$to the right of the . Parallel straight line $P_1{P}_0$and straight line $P_1$The cross product of$Q$$Q$ in line$P_1{P}_0$on the right side; and finally launched $Q$ in line$P_0{P}_2$on the left side. Explanation point $Q$ is outside the triangle.

Therefore, triangle rasterization only needs to traverse each point to determine whether it is inside it. Of course, we can further optimize, because it is obviously not necessary to test every point in the screen, a triangle surface may only occupy a small part of the screen, we can use a bounding box to surround the triangle we want to test, only for the The points in the bounding box are sampled and tested, as shown in the figure below:

5. Aliasing and Aliasing

5.1 Jaggies and aliasing caused by rasterization

Using the above-mentioned triangle rasterization method, the final figure we get is as shown in the figure below:

It can be seen that it is originally a triangle, but after sampling, it is not a triangle at all, and the edges are very uneven and smooth. This phenomenon is in graphics. called aliasing.

Why is it out of shape? Its essence is that the sampling frequency is too low, so that the sampling frequency cannot keep up with the frequency of the image. In layman's terms, the number of samples is too small. You can imagine that if there are enough pixels and the number of samples is large enough, the fineness will be higher, and the aliases will become so small that they cannot be distinguished by the naked eye, so it looks like Flat and smooth.

5.2 How to anti-alias / anti-aliasing

The basic idea of ​​anti-aliasing/anti-aliasing is blurring. For example, we can blur triangles first, and then sample them, as shown in the figure below:

To understand why blurring can anti-aliasing from a deep level, we need to involve time domain and Some knowledge of the frequency domain. As shown in the figure below, the product of the time-domain pixels corresponding to the sampling is equal to the convolution of the corresponding frequency domain and low-frequency filter of the image, both of which can achieve a blurred effect: in short, the solution to the problem is to use finite discrete

pixels to approximate continuous The triangle, two methods adopted by the industry are introduced below.

5.2.1 Super Sampling Anti-Aliasing (SSAA)

This is the most basic anti-aliasing mode. The principle of implementation is to enlarge the image by several times the resolution of the display when rendering. For example, if 2xSSAA is enabled on a resolution of 1024 x 768, the GPU will first render a 2048 x 1536 image and then "stuff it in" Forming in the frame of 1024 x 768, doubling the fineness of the picture, will undoubtedly improve the situation of edge aliasing.

But as we all know, the rendering of high-resolution graphics will greatly consume GPU computing resources, video memory capacity and bandwidth, so SSAA resource consumption is huge, and even the lowest 2x may not be easily bearable.

This method is nothing more than increasing the resolution, that is, increasing the sampling points. As shown in the figure below, each pixel is subdivided into 4 sampling points:


then coloring is performed according to each sampling point, and every time a sampling point is covered, it is colored once. . After obtaining the color of each sampling point in this way, we add up all the color values ​​of the subdivided sampling points inside each pixel point for an average, and use it as the color value of the pixel point after anti-aliasing. The result is as follows:

5.2.2 Multi Sampling Anti-Aliasing (MSAA)

MSAA is an improved version of SSAA. SSAA has to re-render the entire screen at several times the resolution just for edge smoothing, resulting in a great waste of precious graphics card processing resources. Therefore, MSAA was born to improve this situation. The implementation of MSAA is similar to SSAA, the difference is that MSAA only enlarges the edge part of the 3D modeling, not the entire picture. Simply put, the 3D model is composed of a large number of polygons, and MSAA only processes the outermost polygons of the model. , so the burden on the graphics card is greatly reduced.

Although MSAA tends to be easy to use and is very popular, its shortcomings are also obvious

• If there are many unit objects in the picture, the number of edge polygons that need to be processed will naturally increase, and the performance of MSAA will also drop significantly at this time;
• With the same multiple of MSAA, the edge smoothing effect is theoretically the same as that of SSAA, but because only the polygons on the edge are processed, the texture sharpness of the non-edge part is far less than that of SSAA.

Also use the above example to illustrate the difference between MSAA and SSAA. MSAA still divides pixels into multiple sampling points. The difference is that it is no longer necessary to color every time a sampling point is covered, but to count the number of covered sampling points. For example, if two sampling points are covered, you only need to multiply the color value calculated by changing the center of the pixel by 50%, which greatly reduces the amount of calculation, as mentioned above:

6. Depth buffer (Z-Buffer)

In computer graphics, depth buffering is the process of processing image depth coordinates in 3D graphics. This process is usually done in hardware, but it can also be done in software. It is a solution to the visibility problem. The visibility problem is the problem of determining which parts of the rendered scene are visible and which parts are not.

Following the above, after we have processed the rasterization of graphics, we still need to consider the relationship between objects? This is very important. To put it more bluntly, it is necessary to figure out the layer of the object, which object will be blocked by which object, and which object will block which object. Specifically, each pixel may correspond to more than one point on a triangle surface, which point on the triangle surface should be selected for display?

Of course, the pixel closest to the camera is displayed, which requires the use of a depth buffer. (The distance of the 3D object is represented by the Z ZZ axis, so it is also called Z-Buffer), as shown in the figure below, and the depth map is shown on the right: the

specific steps are as follows:

• The Z-Buffer algorithm needs to maintain a depth array for each pixel, which is recorded as zbuffer, and the initial value of each position is set to infinity (that is, it is infinitely far away from the camera);
• Then we traverse each pixel point [x, y] on each triangle surface. If the depth value z of the pixel point is less than the value in zbuffer[x, y], update the value of zbuffer[x, y] to the Point depth value z, and at the same time update the color of the pixel point [x, y] to the color of the point on the triangle surface.

Initialize depth buffer to ∞
During rasterization:
	for(each triangle T)
		for(each sample(x,y,z) in T)
			if(z < zbuffer[x,y])              //目前为止离摄像机最近的
				framebuffer[x,y] = rgb;       //更新颜色
				zbuffer[x,y] = z;             //更新深度

To give a vivid example, the number represents the depth, and the smaller the number, the closer to the camera. We can use the above algorithm to update the pixel color.