1. Introduction to ffmpeg
FFmpeg is a set of open source computer programs that can be used to record, convert digital audio and video, and convert them into streams. Adopt LGPL or GPL license. It provides a complete solution for recording, converting, and streaming audio and video. It contains a very advanced audio/video codec library libavcodec. In order to ensure high portability and codec quality, many codes in libavcodec are developed from scratch. The multimedia video processing tool FFmpeg has very powerful functions including video capture, video format conversion, video capture, and watermarking of videos.
- Video capture function: The ffmpeg video capture function is very powerful. It can not only capture images from a video capture card or USB camera, but also perform screen recording. At the same time, it also supports RTP to transmit the video stream to a streaming media server that supports RTSP, and supports live applications. .
- Video format conversion function: ffmpeg video conversion function. Video format conversion, for example, can convert multiple video formats to flv format, but not video signal conversion. ffmpeg can easily convert between multiple video formats (wma, rm, avi, mod, etc.), for example, it can convert the recorded video avi, etc. into the flv format used by the video website. There are many optional commands during transcoding. Encoder, video duration, frame rate, resolution, pixel format, sampling format, bit rate, cropping options, number of channels, etc. can be freely selected:
- Video screenshot function: For the selected video, the thumbnail of the specified time is captured. Video capture, obtain static images and dynamic images, it is not recommended to capture gif files; because the captured gif files are large and the playback is not smooth.
- Add watermark function to video: use ffmpeg video to add watermark (logo).
2. Common functions of ffmpeg
(1) View video information:
ffmpeg -i video.avi
参数：
-i：表示输入的源视频This command can view the video duration, bit rate, audio encoding method, audio bit rate, video encoding method, video bit rate, frame rate, video resolution and other information.
(2) Cut out segments from the video:
ffmpeg -i output.mp4 -ss 00:00:00 -c copy -to 00:00:10 output_10.mp4
参数：
-ss：指定裁剪的开始时间；
-to：指定裁剪的结束时间；
-c：指定音频和视频的编码方式，这里为copy，表示目标视频和源视频相同，-c可以细分为图像-c:v和音频-c:a；(3) Video segment merging:
ffmpeg -f concat -i filelist.txt -c copy output.avi
参数：
filelist.txt的内容为：
file '001.avi'
file '002.avi'
file '003.avi' (4) Extract audio from video:
ffmpeg -i video.mp4 -vn audio.mp3
参数：
-vn：表示去掉视频(5) Extract silent video from video:
ffmpeg -i video.mp4 -an output.mp4
参数：
-an：表示去掉音频(6) Extract image sequence from video:
ffmpeg -i video.mp4 image%d.jpg
# 上面的命令会生成image1.jpg，image2.jpg，image3.jpg， ... ，imageN.jpg(7) Image sequence and audio are merged into video:
ffmpeg -r 30 -i image%d.jpg -b 200K output.mp4
参数：
-r：指定视频的帧率，注意-r要放在-i前面;
-b: 指定比特率；There are three image formats, jpg, png, and bmp. jpg has the largest compression loss, the smallest image, and the worst quality. Bmp has no compression, the largest image, and png is somewhere in between. Therefore, if we want to synthesize the decomposed image sequence into the original video, it is best to use the bmp format.
- If you want to synthesize MP4 video, the image format can only be jpg, the other two formats cannot be used;
- Images in png and bmp format can be synthesized into avi format video, but not in MP4 format;
(8) Video and audio synthesis video:
ffmpeg -i input.mp4 -i audio.mp3 -c copy output.mp4(9) Adjust the video frame rate:
ffmpeg -i input.avi -c:a copy -r 30 output.mp4
参数：
-r：指定帧率
-c:a: 音频编码方式不变(10) Adjust the video bit rate:
Bit rate is also a parameter that determines the overall quality of audio and video. It determines the number of bits processed per unit of time. The bit rate determines how many bits are needed to process a 1s coded stream.
- The amount of data per second before compression (one byte size can be calculated here) = frame rate × resolution;
- Compression ratio = data volume per second before compression / bit rate;
- File size = bit rate × video duration;
The bit rate is not as big as possible, nor is it as small as possible. The bit rate has a great relationship with the resolution. Generally, the higher the resolution, the higher the bit rate is required. The resolution is small, and only a small bit rate is required.
ffmpeg -i file.avi -b 1.5M file.mp4
参数：
-b：指定比特率，这里-b可以细分为音频比特率-b:a和视频比特率-b:v，-b则表示整体比特率；(11) Adjust the video resolution:
ffmpeg -i input.mp4 -vf scale=-1:360 -c:a copy output.mp4
参数：
-vf:对视频进行过滤，-1表示原视频调整后不会变形- The resolution and bit rate of the narrow screen are configured as follows:
- The resolution and bit rate of the widescreen are configured as follows
(12) Speed up the playback speed of video and audio:
ffmpeg -i input.mp4 -vf "setpts=0.5*PTS" -af "atempo=2.0" output.mp4
参数：
-vf：等价于-filter:v，即对视频进行过滤，这里提高视频为原来的两倍；
-af：等价于-filter:a，即对音频进行过滤，这里提高音频为原来的两倍；3. GPU acceleration of ffmpeg
ffmpeg can use GPU for acceleration, you only need to configure the driver to compile at compile time. For specific tutorials, please refer to here . After installing the GPU version of ffmpeg, we first need to query the hardware accelerators supported by ffmpeg:
ffmpeg -hwaccels
输出：
Hardware acceleration methods:
cuda
cuvid
Then we need to check which encoders and decoders are supported by the hardware accelerator:
ffmpeg -codecs | grep cuvid
输出：
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
configuration: --enable-cuda --enable-cuvid --enable-nvenc --enable-nonfree --enable-libnpp
--extra-cflags=-I/usr/local/cuda-10.0/include --extra-ldflags=-L/usr/local/cuda-10.0/lib64
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m h264_cuvid)
(encoders: h264_nvenc h264_v4l2m2m nvenc nvenc_h264 )
DEV.L. hevc H.265 / HEVC (High Efficiency Video Coding) (decoders: hevc hevc_cuvid )
(encoders: nvenc_hevc hevc_nvenc )
DEVIL. mjpeg Motion JPEG (decoders: mjpeg mjpeg_cuvid )
DEV.L. mpeg1video MPEG-1 video (decoders: mpeg1video mpeg1_v4l2m2m mpeg1_cuvid )
DEV.L. mpeg2video MPEG-2 video (decoders: mpeg2video mpegvideo mpeg2_v4l2m2m mpeg2_cuvid )
DEV.L. mpeg4 MPEG-4 part 2 (decoders: mpeg4 mpeg4_v4l2m2m mpeg4_cuvid ) (encoders: mpeg4 mpeg4_v4l2m2m )
D.V.L. vc1 SMPTE VC-1 (decoders: vc1 vc1_v4l2m2m vc1_cuvid )
DEV.L. vp8 On2 VP8 (decoders: vp8 vp8_v4l2m2m vp8_cuvid ) (encoders: vp8_v4l2m2m )
D.V.L. vp9 Google VP9 (decoders: vp9 vp9_v4l2m2m vp9_cuvid )
All those with "cuvid" or "nvenc" can be seen by the GPU codec provided by CUDA, we can now perform GPU decoding in h264 / hevc / mjpeg / mpeg1 / mpeg2 / mpeg4 / vc1 / vp8 / vp9 format , And GPU encoding in h264/hevc format. Then we can use the -hwaccel parameter to specify the hardware accelerator, specify the video decoder and video encoder for GPU acceleration. such as:
- Compress the video, adjust the bit rate, and the resolution remains unchanged:
ffmpeg -hwaccel cuvid -c:v h264_cuvid -i test.mp4 -c:v h264_nvenc -b 2.5M test_gpu.mp4
参数：
-hwaccel cuvid：指定使用cuvid硬件加速；
-c:v h264_cuvid：使用h264_cuvid进行视频解码；
-c:v h264_nvenc：使用h264_nvenc进行视频编码；
-b 2.5M : 指定视频输出的比特率为2.5M；- Adjust the resolution of the video. At this time, the bit rate should be adjusted accordingly:
ffmpeg -y -hwaccel cuvid -c:v h264_cuvid -i test.mp4 -b 0.96M -vf scale_npp=-1:320 -c:v h264_nvenc test_gpu.mp4
参数：
-hwaccel cuvid：指定使用cuvid硬件加速；
-c:v h264_cuvid：使用h264_cuvid进行视频解码；
-c:v h264_nvenc：使用h264_nvenc进行视频编码；
-b 0.96M : 指定视频输出的比特率为0.96M；
-vf scale_npp=-1:320：进行硬解码，即GPU解码，使用硬件加速时使用该参数，而-vf scale表示使用软解码，即CPU解码，
不使用硬件加速时使用该参数。-1表示原视频调整后不会变形；Reference: https://developer.nvidia.com/ffmpeg