OpenGLES: Camera real-time filter four-grid, nine-grid

I. Overview

Continue learning OpenGLES today

Today, based on the previous blog post "OpenGLES: GLSurfaceView implements Android Camera preview" , OpenGLES is used to implement the camera's four-square grid filter and nine-square grid filter .

2. Four-square grid

First define a few terms:

The normal camera preview implemented in the previous blog post is called: "Big Palace Grid"

What we need to implement now is: " Four-square grid" + "Filter"

Each small square of the four-square grid is called: "Little Palace Grid"

The points in the small grid area are recorded as: (min_x, min_y)

The origin of the small palace grid is recorded as: (min_origin_x, min_origin_y)

The points in the large grid area are recorded as: (max_x, max_y)

The origin of the big palace grid is (0,0) , no special marking is needed

Let’s first look at how to implement the four-square grid , and then look at how to add filters

The small grid (min_x, min_y) is subordinate to the large grid (max_x, max_y). It is just a 1/4 sub-area of (max_x, max_y). What
we have to do is how to convert (min_x, min_y) to correspond to (max_x, max_y)

It is easy to see from the diagram:

1. The large grid is equivalent to the small grid moving in the opposite direction along the x and y axes (min_origin_x, min_origin_y) to (0,,0), and then multiplying the length and width by 2
can also be considered in the reverse direction:
2. Small grid It is equivalent to dividing the length and width of the large palace grid by 2, and then moving along the x and y axes in the positive direction to (min_origin_x, min_origin_y)

Take the "small palace grid in the upper right corner" as an example :

1. The large grid is equivalent to moving the small grid in the upper right corner from (0.5, 0.5) to (0, 0) in the opposite direction along the x and y axes, and then enlarging the length and width by 2 times.

or

2. The small grid in the upper right corner is equivalent to reducing the length and width of the large grid by 1/2, and then moves along the positive direction of the x and y axes to (0.5, 0.5)

What we need to do is how to map some coordinates ( small grid ) in the texture sampling coordinates to all coordinates ( large grid )

So what we want to find is the corresponding relationship of the above "1." , and the reverse "2." is for a more vivid understanding of the process.

According to this method, no matter how complicated the grid division is, as long as we deduce how the small grid can overlap with the "big grid" through translation and scaling , we can find the corresponding relationship.

Therefore, the formula for converting "small palace grid" to "large palace grid" is :

max_x = (min_x - min_origin_x) *2
max_y = (min_y - min_origin_y) *2

(min_origin_x, min_origin_y) can be further formulated by the "several" of the "several grids " to be realized . This is not difficult, so we will not deduce it.

3. Filter

In the fragment shader, create the texture from the texture sampler and texture coordinates

The texture created is actually a vector of color values assigned to the built-in variables.

This is why we need to use the texture sampler and texture coordinates , which is equivalent to the texture sampler obtaining the color value of this coordinate point based on the texture coordinates. All the color values form a colorful picture, which is the texture .

You can get the RGB component of the color value through the color vector

Then you can create various "filter" effects on colors.

The following is the black and white filter code:

//黑白
void blackAndWhite(inout vec4 color){
    float threshold = 0.5;
    float mean = (color.r + color.g + color.b) / 3.0;
    color.r = color.g = color.b = mean >= threshold ? 1.0 : 0.0;
}

You can understand it directly through the code literal:

If the average value of the three components (r, g, b) of the color value sampled by the texture coordinate point is >0.5, it will be assigned the value white, otherwise it will be black.

The following is the final implementation of the black and white, grayscale, reverse, and original image real-time filter shader code of the four-square grid :

The words are easy to understand, so I won’t explain them one by one.

#version 300 es
#extension GL_OES_EGL_image_external_essl3 : require
precision mediump float;

in vec2 texCoord;//纹理坐标，图片当中的坐标点
out vec4 outColor;

uniform samplerExternalOES s_texture;//图片，采样器

//黑白
void blackAndWhite(inout vec4 color){
    float threshold = 0.5;
    float mean = (color.r + color.g + color.b) / 3.0;
    color.r = color.g = color.b = mean >= threshold ? 1.0 : 0.0;
}

//灰度
void grey(inout vec4 color){
    float weightMean = color.r * 0.3 + color.g * 0.59 + color.b * 0.11;
    color.r = color.g = color.b = weightMean;
}

//反向
void reverse(inout vec4 color){
    color.r = 1.0 - color.r;
    color.g = 1.0 - color.g;
    color.b = 1.0 - color.b;
}

void main(){
    float x = texCoord.x;
    float y = texCoord.y;

    //四宫格滤镜
    if (x <= 0.5 && y <= 0.5){
        x = x * 2.0;
        y = y * 2.0;
        outColor = texture(s_texture, vec2(x, y));
        reverse(outColor);
    } else if (x <= 0.5 && y >= 0.5){
        x = x * 2.0;
        y = (y-0.5) * 2.0;
        outColor = texture(s_texture, vec2(x, y));
        blackAndWhite(outColor);
    } else if (x>0.5 &&  y > 0.5){
        x = (x-0.5) * 2.0;
        y = (y-0.5) * 2.0;
        outColor = texture(s_texture, vec2(x, y));
        grey(outColor);
    } else if (x>0.5 &&  y < 0.5){
        x = (x-0.5) * 2.0;
        y = y * 2.0;
        outColor = texture(s_texture, vec2(x, y));
    }
}

Realization effect:

4. Nine-square grid filter

If you have followed the blog till now and implemented the four-square grid filter by yourself, then the nine-square grid filter is just a matter of course.

In the nine-square grid sample code, I did not calculate the implementation one by one, but implemented it through the equally divided range of x and y.

Therefore, we did not add filter effects one by one. The focus was on the realization of Jiugongge.

As mentioned in the previous article, as long as the correspondence between the small grid and the large grid is found , the shader code can be further streamlined based on the " number" of "how many grids " and use loop statements to divide the grids.

Once you understand the principle thoroughly, you will never be able to deviate from it. The code implementation is just a form.

The following is the Jiugongge shader code:

#version 300 es
#extension GL_OES_EGL_image_external_essl3 : require
precision mediump float;

in vec2 texCoord;//纹理坐标，图片当中的坐标点
out vec4 outColor;

uniform samplerExternalOES s_texture;//图片，采样器

void main(){
    float x = texCoord.x;
    float y = texCoord.y;

    //九宫格滤镜
    float x = texCoord.x;
    float y = texCoord.y;

    if (x < 1.0 / 3.0) {
        x = x * 3.0;
    } else if (x < 2.0 / 3.0) {
        x = (x - 1.0 / 3.0) * 3.0;
    } else {
        x = (x - 2.0 / 3.0) * 3.0;
    }
    if (y <= 1.0 / 3.0) {
        y = y * 3.0;
    } else if (y < 2.0 / 3.0) {
        y = (y - 1.0 / 3.0) * 3.0;
    } else {
        y = (y - 2.0 / 3.0) * 3.0;
    }

    outColor = texture(s_texture, vec2(x, y));
}

Realization effect: