Basics of Stencil Test

This article shares the basics of Stencil Test

In the rendering pipeline, stencil testing occurs after fragment shader processing and transparency testing, but before depth testing.

The most common application of template testing is various masks, especially shaped masks, such as Maskcomponents in Unity. The characteristic of these masks is that they can render fragments according to a certain shape (which can be non-rectangular).

For example, we often see avatars of various shapes in games, and Rect Mask 2Dthe component uses Scissor Test, which can specify to draw a fragment in a rectangular area. The circular mask of the template test is shown in the figure.

Template Testing Basics

Similar to the depth test, the stencil test also has a corresponding buffer, namely the stencil buffer (Stencil Buffer), which is used to record the stencil value of all pixels , and the default value is 0.

When rendering an object, we can specify a reference value (Referencce value, called Stencil ID in unity), when rendering a fragment of this object , compare the reference value carried by the fragment with the value in the template cache , according to different comparison methods, if the requirements are met, the test will pass, and then the value in the template cache will be updated according to the set operation. Of course, the operation when the test fails can also be specified.

From the above process, we should pay attention to several key points:

• template value : the value that already exists in the template cache
• Reference value : Before rendering the object, the specified value is set by the program
• Comparison function : A function that determines how to compare two values
• Operation function : define the update operation of the template value after passing or failing the test

The template test is a non-programmable, but configurable pipeline stage. We configure the template test through instructions. In Unity, its basic syntax is:

Stencil
{
    Ref refValue
    Comp always
    Pass keep
    Fail keep
    ZFail keep
    WriteMask 255
    ReadMask 255
}

Explanation of Template Test Syntax in Unity

We Ref refValuespecify the reference value by , the reference value is an integer value, positive or negative.

Then pass Comp xxxto specify the comparison function, the received parameter is the enumeration value of the comparison function , which corresponds to the enumeration in C# UnityEngine.Rendering.CompareFunction, from 0 to 8, respectively (the corresponding enumeration is in brackets):

• 0(Disabled): Turn off the template test, which is equivalent to passing all the tests, and it is found that it is not really closed after testing.
• 1(Never):All failed the test
• 2(Less): pass the test when the value to be compared is less than the value in the cache
• 3(Equal): Pass the test when the value to be compared is equal to the value in the cache
• 4(LessEqual): pass the test when the value to be compared is less than or equal to the value in the cache
• 5(Greater): pass the test when the value to be compared is greater than the value in the cache
• 6(NotEqual): pass the test when the value to be compared is not equal to the value in the cache
• 7(GreaterEqual): pass the test when the value to be compared is greater than or equal to the value in the cache
• 8(Always): All tests passed, default value

Then, through various conditions of passing or failing the test, specify the update operation of the template value, respectively:

• Pass operation: The operation after both the template test and the depth test pass
• Fail operation: The operation after both the stencil test and the depth test fail
• ZFail operation: The operation after the stencil test passes but the depth test fails

Their parameters are all of the same type, that is, the enumeration value of the operation function , corresponding to the enumeration in C# UnityEngine.Rendering.StencilOp, from 0 to 7, representing:

• 0(Keep): keep the value in the template cache unchanged
• 1(Zero): Set the value in the template cache to 0
• 2(Replace): Replace the value in the template cache with the reference value
• 3(IncrementSaturate): Increase the stencil buffer value, limited to the largest representable unsigned value
• 4(DecrementSaturate): Decrease the stencil buffer value, the minimum limit is 0
• 5(Invert): Bitwise negate the stencil buffer value
• 6(IncrementWrap): Similar to IncrementSaturate, but it will be reset to 0 if it continues to increase after reaching the maximum
• 7(DecrementWrap): Similar to DecrementSaturate, except that it will be reset to the largest representable unsigned value if it continues to decrease after reaching the minimum

The last ReadMaskand WriteMasktwo masks are additional processing of the template value and the reference value, the value is an integer value between 0-255, ReadMaskwhich means that after reading the template value, it is bitwise ANDed with the mask and then combined with the reference value For comparison, it WriteMaskmeans that before using the reference value to update the template value, it is updated after the template value and the mask are bitwise ANDed. Generally, both masks are set to 255, which means that no additional processing is performed.

In Unity, it can be set through the property panel, but after testing, using variables to set template parameters cannot close the template test:

_Stencil ("Stencil ID", Float) = 0
[Enum(UnityEngine.Rendering.CompareFunction)] _StencilComp ("Stencil Comparison", Int) = 8
[Enum(UnityEngine.Rendering.StencilOp)] _StencilOp ("Stencil Operation", Int) = 0
_StencilWriteMask ("Stencil Write Mask", Range(0, 255)) = 255
_StencilReadMask ("Stencil Read Mask", Range(0, 255)) = 255

Stencil
{
    Ref [_Stencil]
    Comp [_StencilComp]
    Pass [_StencilOp]
    ReadMask [_StencilReadMask]
    WriteMask [_StencilWriteMask]
}

The effect is as shown in the figure:

The above is the basic content of template testing. In the next article, we will use template testing to simulate Maskcomponents in Unity, and use another method to solve the jagged problem. I hope it will be helpful to everyone.