FFmpeg commands: from beginner to proficient | ffmpeg filter (filter/filter)
FFmpeg commands: from beginner to proficient | ffmpeg filter (filter/filter)
This section mainly introduces ffmpeg filter, listing some commonly used filters and how to use them.
There are many types of ffmpeg filters, which is also the most difficult point in learning ffmpeg.
ffmpeg filter basic built-in variables
| variable | illustrate |
|---|---|
| t | timestamp in seconds, or NAN if the input time is unknown |
| n | Enter the sequential number of the frame, starting from 0 |
| pos | The position of the input frame, or NAN if unknown |
| w | Input video frame width |
| h | The height of the input video frame |
Video cropping
Input image size: iw * ih, starting coordinates (x, y), cropped output size: ow * oh.
Display directly:
ffplay -i input.jpg -vf crop=ow:oh:x:y
Output picture:
ffmpeg -i input.jpg -vf crop=ow:oh:x:y output.jpg
Crop the middle area with size 100 * 100:
ffmpeg -i input.jpg -vf crop=100:100 output.jpg
Crop the middle area to 2/3 of the input video:
ffmpeg -i input.flv -vf crop=2/3*in_w:2/3*in_h output.flv
Crop the middle area to a square with the height of the input video:
ffmpeg -i input.flv -vf crop=out_w=in_h output.flv
or
ffmpeg -i input.flv -vf crop=in_h output.flv
Example 1:
ffmpeg -i input.jpg -vf crop=iw/3:ih:iw/3:0 output.jpg
Starting from coordinates (iw/3, 0), crop out a picture with width iw/3 and height ih, name it output.jpg and save it.
input.jpg：
output.jpg：
Example 2:
ffplay -i input.jpg -vf crop=iw/3:ih/2:0:0
Starting from coordinates (0, 0), a picture with width iw/3 and height ih/2 is cropped and displayed:
text watermark
Adding text watermarks to videos requires many preparations. You need to have relevant files for text font processing. When compiling FFmpeg, you need to support FreeType, FontConfig, and iconv. You need to have relevant fonts in the system. You can add pure letter watermarks to FFmpeg. Use the drawtext filter for support. Let’s take a look at the filter parameters of drawtext:
| parameter | type | illustrate |
|---|---|---|
| text | string | Character content |
| textfile | string | A file that stores text content |
| box | Boolean value | Text area background box, default is false |
| boxcolor | color | Display the color of the font area block |
| font | string | Font name, default is Sans font |
| fontsize | integer | font size |
| x and y | string | The position where the text watermark appears, with the upper left corner of the video as the starting coordinate, and the default is 0 |
| alpha | floating point number | Transparency, a floating point number with a value of 0 ~ 1, the default is 1 |
Example 1: Add text watermark to the upper left corner of the video
ffplay -i input.mp4 -vf "drawtext=fontsize=100:fontfile=FreeSerif.ttf:text='hello world':x=20:y=20"
Set the font color to green:
ffplay -i input.mp4 -vf "drawtext=fontsize=100:fontfile=FreeSerif.ttf:text='hello world':fontcolor=green"
If you want to adjust the position of the text watermark display, just adjust the values of the x and y parameters:
ffplay -i input.mp4 -vf "drawtext=fontsize=100:fontfile=FreeSerif.ttf:text='hello world':fontcolor=green:x=400:y=200"
Modify transparency:
ffplay -i input.mp4 -vf "drawtext=fontsize=100:fontfile=FreeSerif.ttf:text='hello world':fontcolor=green:x=400:y=200:alpha=0.5"
You can also add a box to the text watermark, and then add a background color to the box:
ffplay -i cuc_ieschool.mkv -vf "drawtext=fontsize=40:fontfile=FreeSerif.ttf:text='hello world':fontcolor=green:box=1:boxcolor=yellow"
Effect:
Note: The entire filter is enclosed in quotation marks, and each parameter is separated by a colon.
Example 2: Use local time as watermark content
Sometimes text watermarks want to use local time as the watermark content. This can be done with some special usage in the drawtext filter. The current local time is displayed in text in the format of year, month, day, hour, minute and second.
ffplay -i cuc_ieschool.mkv -vf "drawtext=fontsize=20:fontfile=FreeSerif.ttf:text='%{localtime\:%Y\-%m\-%d %H-%M-%S}':fontcolor=green:box=1:boxcolor=yellow"
Effect:
Note: The time will be displayed in real time and constantly changing.
When using ffmpeg to transcode and store it into a file, you need to add -re (which means watermarking according to the timestamp, because it is transcoding, there will be multiple frames in 1s, and there will be exceptions when displayed according to the current time), otherwise the time is wrong.
ffmpeg -re -i input.mp4 -vf "drawtext=fontsize=30:fontfile=FreeSerif.ttf:text='%{localtime\:%Y\-%m\-%d %H-%M-%S}':fontcolor=green:box=1:boxcolor=yellow" output.mp4
In some scenes, it is necessary to display the watermark regularly or not to display the watermark regularly. This method can also be processed with the drawtext filter. Use drawtext in conjunction with enable. For example, a text watermark is displayed every 3 seconds:
ffplay -i input.mp4 -vf "drawtext=fontsize=60:fontfile=FreeSerif.ttf:text='test':fontcolor=green:box=1:boxcolor=yellow:enable=lt(mod(t\,3)\,1)"
Note: You need to add -re when using ffmpeg to transcode and store to a file, otherwise the time will be wrong.
Expression reference: FFmpeg Utilities Documentation 3 Expression Evaluation
- lt(x, y): Return 1 if x is lesser than y, 0 otherwise. Returns 1 if x is less than y, otherwise returns 0
- mod(x, y)：Compute the remainder of division of x by y。x 对 y 取余。
Example 3: Marquee effect
Marquee means that the text watermark fluctuates on the video interface, and the horizontal x, y coordinates are dynamically modified according to the mod formula to achieve the marquee effect.
ffplay -i input.mp4 -vf "drawtext=fontsize=30:fontfile=FreeSerif.ttf:text='helloworld':x=mod(100*t\,w):y=abs(sin(t))*h*0.7"
Effect: The text watermark appears cyclically in a sine function line.
Modify font transparency and font color:
ffplay -i input.mp4 -vf "drawtext=fontsize=40:fontfile=FreeSerif.ttf:text='liaoqingfu':x=mod(50*t\,w):y=abs(sin(t))*h*0.7:alpha=0.5:fontcolor=white:enable=lt(mod(t\,3)\,1)"
Effect:
Picture watermark
In addition to adding text watermarks to videos, FFmpeg can also add image watermarks, video tickers, etc. to videos.
To add image watermarks to videos, you can use the movie filter. Let’s get familiar with the parameters of the movie filter:
| parameter | type | illustrate |
|---|---|---|
| filename | string | Enter the file name, which can be a file, protocol, device, etc. |
| format_name or f | string | The packaging format of the input file |
| stream_index | integer | The stream index number of the input file |
| seek_point or sp | floating point number | The time position of the input stream |
| streams or s | string | Stream information for multiple input streams |
| loop | integer | Cycles |
| discontinuity | time difference | Support for jittering timestamp differences |
ffmpeg -i input.mp4 -vf "movie=logo.png[watermark];[in][watermark]overlay=x=10:y=10[out]" output.mp4
parameter:
- -i: original video file path
- Watermark image path: logo.png
- Watermark position: (x, y) = (10, 10) <= (left, top), 10 pixels from the left and top
- Output file path: output.mp4
- in: represents input.mp4, the name can be randomly determined
- watermark: represents the logo.png to be superimposed. The name can be randomly determined.
How to determine the location of image watermark?
| parameter | illustrate |
|---|---|
| main_w | Video single frame picture width |
| main_h | Video single frame picture height |
| overlay_w | Video single frame picture width |
| main_h | Video single frame picture height |
Correspondingly, you can set the overlay parameter to the following value to change the position of the watermark image:
| Image watermark location | overlay value |
|---|---|
| upper left corner | 10:10 |
| upper right corner | main_w-overlay_w-10:10 |
| lower left corner | 10:main_h-overlay_h-10 |
| lower right corner | main_w-overlay_w-10:main_h-overlay_h-10 |
Tips: If the watermark image has a transparent background, the effect will be better.
Similarly, we can also add a marquee effect:
ffplay -i input.mp4 -vf "movie=logo.png[watermark];[in][watermark]overlay=x=mod(50*t\,main_w):y=abs(sin(t))*h*0.7[out]"
picture within picture
Picture-in-picture, as the name suggests, is a video being played with another video superimposed on it.
When playing back streaming media files using FFmpeg, sometimes you need to use the picture-in-picture effect.
In FFmpeg, multiple video streams, multiple multimedia capture devices, and multiple video files can be merged into one interface through overlay to generate a picture-in-picture effect.
In the previous use of filters, and even in the future use of filters, most of the processing related to video operations will be used in conjunction with the overlay filter, especially in layer processing and merging scenarios. Let’s learn about it below. Here are the parameters of overlay:
| parameter | type | illustrate |
|---|---|---|
| x | string | x coordinate |
| y | string | y coordinate |
| eof_action | integer | 遇到 eof 表示时的处理方式。repeat（值为0）：重复前一帧，子画面保存前一帧；endcall（值为1）：停止所有的流，主画面和子画面全部停止；pass（值为2）：保留主画面，子画面关闭。默认为repeat（值为0） |
| shortest | 布尔值 | 值为 true 时，最短的视频终止时全部视频终止。默认值为 false |
| format | 整数 | 设置 output 的像素格式。有以下几种格式：yuv420（值为0）、yuv422（值为1）、yuv444（值为2）、rgb（值为3）.默认值为yuv420（值为0） |
显示画中画效果：
ffplay -i input.mp4 -vf "movie=sub_320x240.mp4[sub];[in][sub]overlay=x=20:y=20[out]"
图解：
参数：
- [sub]：表示要叠加的子画面
- [in]：表示输入的视频
- [in][sub]overlay=x=20:y=20[out]：表示输入和 sub 子画面叠加，叠加的位置由 x, y 决定，out 表示输出，sub 子画面的名字可以随意修改
效果：
子画面停止退出显示，主画面正常播放：
ffplay -i input.mp4 -vf "movie=sub_320x240.mp4[sub];[in][sub]overlay=x=20:y=20:eof_action=2[out]"
最短的视频播完则整个画面停止播放：
ffplay -i input.mp4 -vf "movie=sub_320x240.mp4[sub];[in][sub]overlay=x=20:y=20:shortest=1[out]"
重新设定子画面尺寸:
ffplay -i input.mp4 -vf "movie=sub_320x240.mp4,scale=640x480[sub];[in][sub]overlay=x=20:y=20[out]"
跑马灯效果：
ffplay -i input.mp4 -vf "movie=sub_320x240.mp4[test];[in][test]overlay=x=mod(50*t\,main_w):y=abs(sin(t))*main_h*0.7[out]"
视频多宫格处理
视频除了画中画显示，还有一种场景为以多宫格的方式呈现出来，除了可以输入视频文件，还可以输入视频流、采集设备等。
从前文中可以看出进行视频图像处理时，overlay 滤镜为关键画布，可以通过 FFmpeg 建立一个画布，也可以使用默认的画布。
如果想以多宫格的方式展现，则可以自己建立一个足够大的画布，下面就来看一下多宫格展示的例子：
ffmpeg -i 1.mp4 -i 2.mp4 -i 3.mp4 -i 4.mp4 -filter_complex "nullsrc=size=640x480[base];[0:v] setpts=PTS-STARTPTS,scale=320x240[upperleft];[1:v]setpts=PTS-STARTPTS,scale=320x240[upperright];[2:v]setpts=PTS-STARTPTS, scale=320x240[lowerleft];[3:v]setpts=PTS-STARTPTS,scale=320x240[lowerright];[base][upperleft]overlay=shortest=1[tmp1];[tmp1][upperright]overlay=shortest=1:x=320[tmp2];[tmp2][lowerleft]overlay=shortest=1:y=240[tmp3];[tmp3][lowerright]overlay=shortest=1:x=320:y=240" out.mp4
看到这么长的命令可能已经晕了，其实将命令拆分解析一下就很简单了，对命令进行拆分解析：
ffmpeg -i 1.mp4 -i 2.mp4 -i 3.mp4 -i 4.mp4 #所有输入
-filter_complex "nullsrc=size=640x480[base]; #创建了一个base的画面，大小640*480
[0:v] setpts=PTS-STARTPTS,scale=320x240[upperleft]; #视频1命令为upperleft，缩放为320x240
[1:v]setpts=PTS-STARTPTS,scale=320x240[upperright]; #视频2命令为upperright，缩放为320x240
[2:v]setpts=PTS-STARTPTS, scale=320x240[lowerleft]; #视频3命令为lowerleft，缩放为320x240
[3:v]setpts=PTS-STARTPTS,scale=320x240[lowerright]; #视频4命令为lowerright，缩放为320x240
[base][upperleft]overlay=shortest=1[tmp1]; # [base][upperleft]叠加生成[tmp1]，upperleft没设置就是叠加在(0,0)位置
[tmp1][upperright]overlay=shortest=1:x=320[tmp2]; #[tmp1][upperright]叠加生成[tmp2]，叠加在（320,0）位置
[tmp2][lowerleft]overlay=shortest=1:y=240[tmp3]; #[tmp2][lowerleft]叠加生成[tmp3]
[tmp3][lowerright]overlay=shortest=1:x=320:y=240" #[tmp3][lowerright]叠加生成最终的输出
out.mp4
说明：
- 1.mp4、2.mp4、3.mp4、4.mp4 为文件路径
- out.MP4 为输出文件路径
- 通过 nullsrc 创建 overlay 画布，画布大小 640:480
- 使用[0:v][1:v][2:v][3:v]将输入的 4 个视频流去除，分别进行缩放处理
- 基于 nullsrc 生成的画布进行视频平铺，命令中自定义 upperleft, upperright, lowerleft, lowerright 进行不同位置平铺。
图解：
效果：
When there is no video in a part of the frame, that background appears green. As follows:
