Essential Reading Notes for Getting Started with Unity Shader Seventeen

foreword

Here are the notes in the edge detection section. Edge detection uses some basic contents of image science, including convolution kernel, Sobel operator and so on. The whole process is relatively clear.

edge detection algorithm

After obtaining the color buffer for screen rendering, convolve the pixels in the screen with the Sobel operator, and the result of the convolution is a derivative of the current pixel. It may be difficult to understand the derivative here, but it should be easier to understand the degree of change. If the color mutation here is severe, the degree of change will be larger. If the color change is relatively flat, the degree of transformation will be smaller, and the calculation result will be smaller.

Here some Sobel operators and image convolution operations are required.

$\begin{bmatrix} -1 & -2 & -1\\ 0 & 0 & 0\\ 1 & 2 & 1 \end{bmatrix} =Gx$
$\begin{bmatrix} -1 & 0 & 1\\ -2 & 0 & 2\\ -1 & 0 & 1 \end{bmatrix} =Gy$

You may have doubts when reading and writing the code. The part of Gx with value written in the text is clearly above the column, the second column is 0, the other columns have values, and Gy is the second row. 0. But in the code, Gx Seems to be the opposite of Gy.
In the book Image Processing (Gonzalez), Gx and Gy are described as follows: the
operator is multiplied by the pixel, the difference between the third row and the first row is close to the differentiation in the x direction, the third column and The difference between the first columns is close to the differential in the y direction. (P107, Edition 2);

At the same time, there is another problem in coding. Isn't it necessary to turn the operator 180 degrees in the convolution operation, and then multiply and add it? How to code and directly correspond to the multiplication of the camera without rotation?

Answer: It is mentioned in (Image Processing (Gonzalez) P92, Edi 2) that linear spatial filtering is similar to the concept of convolution in the frequency domain, so linear spatial filtering is also called "convolution of mask and image". "Filter mask" is also called "convolution template", "convolution kernel".

That is to say, the convolution of the image and the operator is actually a linear spatial filter, and the expression form of this filter is the corresponding multiplication and addition.

Here is the expression for a linear filter mask of size mxn:

$g\left ( x,y \right )=\sum_{s = -a}^{a}\sum_{t = -b}^{b}w\left ( s,t \right )f\left ( x+s,y+t \right )$
Seeing this, you should know why it is calculated like this.

edge detection

The last wave of edge detection code:

Shader "Custom/8/8.1.2" {//边缘检测
    Properties{
        _MainTex("Main Texture",2D) = "white"{}
        _EdgeColor("Edge Color",color) = (1,1,1,1)
        _BackgroundColor("Background",Color) = (1,1,1,1)
        _EdgeOnly("edgeOnly",float) = 1
    }
    SubShader{

        Pass{

            ZTest Always
            ZWrite Off
            Cull Off

        CGPROGRAM

#pragma vertex vert
#pragma fragment frag
#include "UnityCG.cginc"

        sampler _MainTex;
        float4 _MainTex_TexelSize;//纹素
        float4 _EdgeColor;
        float4 _BackgroundColor;
        float _EdgeOnly;

        struct v2f {
            float4 pos : SV_POSITION;
            float2 uv[9] : TEXCOORD0;
        };


        fixed luminance(fixed3 color)
        {
            return 0.2125 * color.r + 0.7154 * color.g + 0.0721 * color.b;
        }
        half sobel(v2f i) {

            //const half Gx[9] = { -1,-2,-1,
            //  0,0,0,
            //  1,2,1 };
            //const half Gy[9] = { -1,0,1,
            //  -2,0,2,
            //  -1,0,1 };

            const half Gx[9] = { -1,-2,-1,
                0,0,0,
                1,2,1 };
            const half Gy[9] = { -1,0,1,
                -2,0,2,
                -1,0,1 };
            float4 texColor;
            float edgex = 0;
            float edgey = 0;
            for (int it = 0; it < 9; it++)
            {
                texColor = luminance(tex2D(_MainTex, i.uv[it]));
                edgex += texColor * Gx[it];
                edgey += texColor * Gy[it];
            }
            return 1 - abs(edgex) - abs(edgey);
        }

        v2f vert(appdata_img v) {
            v2f o;
            o.pos = UnityObjectToClipPos(v.vertex);
            //求纹素
            float2 uv = v.texcoord;
            o.uv[0] = uv + _MainTex_TexelSize.xy * half2(-1, -1); //左上
            o.uv[1] = uv + _MainTex_TexelSize.xy * half2(0, -1); //左上
            o.uv[2] = uv + _MainTex_TexelSize.xy * half2(1, -1); //左上
            o.uv[3] = uv + _MainTex_TexelSize.xy * half2(-1, 0); //左上
            o.uv[4] = uv + _MainTex_TexelSize.xy * half2(0, 0);
            o.uv[5] = uv + _MainTex_TexelSize.xy * half2(1, 0); //左上
            o.uv[6] = uv + _MainTex_TexelSize.xy * half2(-1, 1); //左上
            o.uv[7] = uv + _MainTex_TexelSize.xy * half2(0, 1); //左上
            o.uv[8] = uv + _MainTex_TexelSize.xy * half2(1, 1); //左上

            return o;
        }
        fixed4 frag(v2f i) :SV_Target{
            half edge = sobel(i);
            fixed4 withEdgeColor = lerp(_EdgeColor, tex2D(_MainTex, i.uv[4]), edge);
            fixed4 onlyEdgeColor = lerp(_EdgeColor, _BackgroundColor, edge);
            return lerp(withEdgeColor, onlyEdgeColor, _EdgeOnly);
        }
            ENDCG
        }
    }
    FallBack Off
}

There is nothing difficult about the shader code, the only thing that needs to be said is to avoid if and loop statements in the shader.