OpenGL, transition effects in video editing

Transition Introduction

What effect is the transition?

Transition effects, transition effects is simply convergence between the two videos.

Now more and more players shooting vlog, if the video is not one or two cool transition effects, are embarrassed to come up with a cool.

So how does the transition effect in the video editing software?

Here are examples of the use of OpenGL to achieve a small video transitions, we can custom GLSL be achieved through different transition effects.

On the Android platform to demonstrate, but in fact whether it is Android or iOS, realization of the principle is the same.

After the first to have two videos, video A and video B, the first player to play video A video B, the middle of a process known as C, C is the video A, B make the transition animation time period.

As follows:

Player in chronological order, from the A - B play>, so that there is a transition effect -> C.

Video transitions, we must first have a video, directly from the video decoding A, B in the current frame and displayed on the screen by OpenGL like, if you are not familiar with this operation, you can see my number [public paper Talking about the history of the article], has written relevant content.

Here in place of the video picture to A, B in the decoded frame.

The final results are as follows:

Realization explain

Analog video rendering play

Analog video is 30 fps, and with 30 ms intervals RxJava triggered once OpenGL rendering.

Observable
        .interval(30, TimeUnit.MILLISECONDS)
        .subscribeOn(AndroidSchedulers.mainThread())
        .subscribe {
            mGLSurfaceView.requestRender()
        }

In addition, if the constantly changing pictures in the video A stage play, which is updated texture content, equivalent to play in a real decoding video.

Of course, these operations just to get closer to this small example of true video transitions, the important thing is how to achieve Shader effects of transition.

First, the transition time to have two textures as input, then you must define two sampler2Dwere sampled.

varying vec2 vTextureCoord;//接收从顶点着色器过来的参数
uniform sampler2D sTexture1;
uniform sampler2D sTexture2;

Wherein sTexture1corresponding to the video content A, sTexture2corresponding to a B video content.

vTextureCoord Texture coordinates corresponding to the transfer from the vertex shader, video A and video B need to use the texture coordinates.

This time, just call texture2D method can get video content A and B of the video.

// 得到视频 A 的内容
texture2D(sTexture1,vTextureCoord)
// 得到视频 B 的内容
texture2D(sTexture2,vTextureCoord)

要注意的是这里说得到视频 A/B 的内容，是得到纹理坐标对应的图像内容。也就是说如果纹理坐标是 [0,1] 范围内，那么可以得到视频 A/B 的全部图像内容。如果坐标是 [-0.5,0.5] 那么只能采样得到一半内容了。

转场效果实现

混合函数 mix

由于转场效果是需要视频 A 和视频 B 进行叠加混合的，而 GLSL 内嵌了 mix 函数进行调用。

对于 GLSL 中有哪些内嵌的函数可以直接调用的，可以参考写过的文章记录：

OpenGL ES 2.0 着色器语言 GLSL 学习

mix 函数的声明如下：

genType mix(genType x,genType y,float a) // 其中 genType 泛指 GLSL 中的类型定义

它的主要功能是使用因子 a 对 x 与 y 执行线性混合，既返回 x * (1-a) + y * a 。

现在，通过 texture2D 能得到视频帧内容，通过 mix 能进行视频帧混合叠加，那么就可以得到最后转场视频帧了。

vec4 color = mix(texture2D(sTexture1,vTextureCoord),texture2D(sTexture2,vTextureCoord),a);

渲染进度控制

似乎到这里就可以大功告成了，实际上才刚刚完成了一半~~~

要知道转场效果是随着时间来播放的，就上面的例子中，转场时间内，一开始都是视频 A 的内容，然后视频 A 逐渐减少，视频 B 逐渐增多，到最后全是视频 B 内容，在我们的 Shader 中也要体现这个时间变化的概念。

在 Shader 中定义 progress 变量，代表转场的播放进度，进度为 0 ~ 1.0 之间。

uniform float progress;

同时在每一次渲染时更新 progress 变量的值。

GLES20.glUniform1f(muProgressHandle, mProgress);

当 progress 为 0 时代表转场刚刚开始，当 progress 为 1 时代表转场已经结束了。

if (mProgress >= 1.0f) {
    mProgress = 0.0f;
} else {
    mProgress += 0.01;
}

这里 progress 每次递增 0.01，完成一次转场就需要 100 次渲染，每次渲染间隔 30ms，那么一次转场动画就是 3000ms 了，当然这个可以自己调节的。

画面绘制

再回到 mix 函数的参数 a ，这个参数起到了随时间调节转场混合程度的作用。当 a = 0 时，全是视频 A 的内容，当 a = 1 时，全是视频 B 的内容。

如上图所示，在转场动画的某一帧，左侧是视频 A 的内容，因为此时 a = 0，右侧是视频 B 的内容，此时 a = 1 。

可以看到在一次渲染绘制内 a 既要能等于 0 ，还要能等于 1 ，这个是怎么实现的呢?

事实上我们说的一次渲染绘制，通常指 OpenGL draw 方法的一次调用，但是在这一次调用里，还是有很多步骤要执行的。

OpenGL 渲染管线会先执行顶点着色器，然后光栅化，再接着就是片段着色器，片段着色器会根据纹理坐标采样纹理贴图上的像素内容进行着色，因此片段着色器在管线中会多次执行，针对每个像素都要进行着色。

上面图像的小方块就好比一个像素，每个像素都要执行一个片段着色器。

首先，肯定所有的像素都要进行着色的。左侧方块采样视频 A 的纹理进行着色，右侧方块采样视频 B 的纹理进行着色。

回到如下代码：

mix(texture2D(sTexture1,vTextureCoord),texture2D(sTexture2,vTextureCoord),a);

只要保证绘制左侧时 a = 0，绘制右侧时 a = 1 就行了。这里可以通过移动纹理坐标来控制 a 的值。

vec2 p = vTextureCoord + progress * sign(direction);
float a = step(0.0,p.y) * step(p.y,1.0) * step(0.0,p.x) * step(p.x,1.0);

OpenGL 中定义纹理坐标范围是 [0 ~ 1] ，可以将范围右移 0.5 ，从而变成 [0.5 ~ 1.5] ，此时纹理坐标一半位于规定范围内，一半超出界外了。

这样就可以通过对当前像素小方格对应的纹理坐标的 x，y 值运用 step 函数进行判断是否在界内，就可以决定是采样视频 A 还是视频 B 的图像了。

当每次刷新 progress 时，就向右移一小段距离，视频 A 随着右移而变少，视频 B 变多，这样就是实现了转场效果。

不知道这个简单的例子有没有让你想到些什么？

对的，没错，就是升职加薪，走向巅峰必备的 PPT 技能，这种视频转场的实现效果就和我们在编辑 PPT 动画时添加的一样。

And this is quite simple, you want to do some of the bells and whistles of transition effects, you can refer to the lack of inspiration inside of the PPT animation.

Further, we can make some conclusions for classification transitions, such as is used in the example image, understood to be the last frame of a first video display and video B A do display transition effect, transition effects such people actually used is relatively small, most of them last a video a video some time ago to do a video B transition effects.

And therefore can be classified transition effects:

A video frame is the last frame of the first video animated transitions make B
Finally, a video A to video B done some time ago video transition animation
A video period last video frame of the first video B animated transitions make
A final video for some time video and video B some time ago to do video animation transitions

The principle of these four classifications fact, almost all, if it is a video, then updates the corresponding texture during video playback.

Just explain more about the use of OpenGL to achieve transition effects in video editing, and through this article I hope everyone can master the basic realization of the principle of transition.

The sample code used in the text, I can focus on the public micro-channel number] [On paper, reply "transition" to ~~~