Getting started with FFmpeg and questions

1. Introduction to ffmpeg

FFmpeg is a free, open-source, cross-platform audio and video processing tool designed to provide high-quality, high-speed multimedia processing. FFmpeg is designed to stream audio and video.

FFmpeg has become one of the most powerful and comprehensive audio and video processing tools recognized in the industry, and has been widely used. Most audio and video related applications and tool libraries directly or indirectly refer to FFmpeg.

The development of FFmpeg began in 2000, initially initiated by French programmer Fabrice Bellard, and later more and more open source contributors participated. Currently FFmpeg is maintained and developed by a team of several developers.

2. ffmpeg author and team

Fabrice Bellard is a famous French computer programmer, known in the industry for founding FFmpeg, QEMU and other projects.

His profile website is bellard.org/.

It lists several projects he created. He has a wide range of interests and has dabbled in many different fields, making him a true master of programming.

3. Splitting and merging of ffmpeg project

In 2011, the FFmpeg project experienced internal disagreements, and some developers chose to leave the FFmpeg project and create the Libav project. Many people dabbling in the field of audio and video processing get confused between the two projects due to the similarity of the code bases and the competition between the two projects.

The differences between the two projects resulted in less acceptance of Libav by the community and caused many users to fall back to FFmpeg.

In 2019, the Libav project announced the official end of development and support, and advised all users to move to FFmpeg.

In the years after the split, the codes of the two projects have diverged considerably, and each has developed its own features and architecture. Therefore, the difference between the old and new versions of ffmpeg is exacerbated after the merger.

4. Examples of functions and commands of ffmpeg

# 从视频中导出音频
ffmpeg -i input.mp4 output.mp3
​
# 截取音频片段
ffmpeg -i input.mp3 -ss 04:10 -t 16 output.mp3
​
# 截取视频片段
ffmpeg -i input.mp4 -ss 04:10 -t 16 -c copy -avoid_negative_ts make_zero output.mp4
​
# 将两个 flv 文件合并
ffmpeg -f concat -i input.txt -c copy xj.flv
​
## input.txt 中的内容
file 'xj1.flv'
file 'xj2.flv'
​
# 将本地视频以 rtmp 协议推流
ffmpeg -re -i ./demo.mp4 -c copy -f flv rtmp://publish.com/live/demo123456
​
# 将在线 flv 流保存到本地
ffmpeg -i http://demo.com/input.flv -c copy dump.flv
​
# 在左上角0,0点添加水印
ffmpeg -y -i tmp.mp4 -i green.png -filter_complex 'overlay=x=0:y=0' output.mp4
​
# 使用华文行楷字体，64字号，红色字，在右上角写水印文字，main_w-text_w-10 就是视频宽度减文字宽度再减 10
ffmpeg -y -i source.mp4 -vf "drawtext=fontfile=STXINGKA.ttf: text='水印文字':x=main_w-text_w-10:y=10:fontsize=64:fontcolor=red:shadowy=2" output.mp4
​
# 将图片转为视频
ffmpeg -loop 1 -f image2 -i test2.png -vcodec libx264 -r 30 -t 3 test2.mp4
​
# 替换视频中的音频轨
ffmpeg -i input.mp4 -i trll.mp3 -map 0:v -map 1:a -c:v copy -shortest -y output.mp4
​
# 绿幕抠像后合并
ffmpeg -i jmfst.mp4 -i greentiger.mp4 -shortest -filter_complex "[1:v]chromakey=0x16ff0a:0.1:0.0[ckout];[0:v][ckout]overlay[out]" -map "[out]" -y output.mp4
​
# 基于 mask 抠像后合并
ffmpeg -i video.mp4 -i matte.mp4 -i background.mp4 -filter_complex '[1][0]scale2ref[mask][main];[main][mask]alphamerge[vid];[2:v][vid]overlay[out]' -map [out] complete.mp4

5. Version differences of ffmpeg command line parameters

When we inquire about how to use the FFmpeg command line on the Internet, we may find that the same effect may be written differently.

For example, -c:v copy -c:a copyin , -c:v and -c:a are new versions, and the corresponding old versions are -vcodec and -acodec.

There are many similar parameter differences between the old and new versions, for example:

• -frames:vand -vframes: both indicate the number of video frames to process. -frames:vFor the new version of writing, -vframesfor the old version of writing.

• -frames:aand -aframes: both indicate the number of audio frames to process. -frames:aFor the new version of writing, -aframesfor the old version of writing.

• -arand -sample_rate: are both used to set the audio sample rate (that is, the number of samples per second). -arFor the new version of writing, -sample_ratefor the old version of writing.

• -acand -channels: are both used to set the number of audio channels. -acFor the new version of writing, -channelsfor the old version of writing.

• -fand -format: are both used to specify the output file format. -fFor the new version of writing, -formatfor the old version of writing.

• -tand -time_limit: are both used to set processing time limits. -tFor the new version of writing, -time_limitfor the old version of writing.

Although there are differences in the command line parameter options of the old and new versions, their purpose and effect are the same. If you are using the latest version of FFmpeg, it is recommended to use the new version as much as possible to maintain compatibility. Because there is no guarantee that FFmpeg will not remove parameters from older versions in the future.

6. ffmpeg filter

ffmpeg filter official documentation ffmpeg.org/ffmpeg-filt... Use ffmpeg -filtersto view all supported filters.

ffmpeg itself has a large number of built-in filters, more than 100 audio filters and more than 200 video filters, as well as various third-party filters.

In FFmpeg, filter can be understood as a concept of data processing pipeline. Similar to the pipe (pipe) on Unix/Linux, it converts and modifies the input data stream through one or more data processors (filters), and finally generates the specified output data stream.

The filter in FFmpeg is mainly used to process and modify audio and video media streams, and can implement many common operations, such as cropping, rotating, scaling, text overlay, etc. By connecting multiple different filters in series, a complex data processing flow can be built to achieve more advanced and specific functional requirements.

Similar to the pipeline on Unix/Linux, in FFmpeg, the basic working principle of the filter is that the input data block is sent to the first filter, and the output is generated after the processing of the filter, and then the output continues to be processed as the input of the next filter , until the end of the last filter, the output is written to the target file or passed to other data processors. This forms a data pipeline, in which each filter represents a specific data processing operation.

In short, although the filter in FFmpeg is not a pipeline in the strict sense, it can be considered that it has many similar concepts and implementation methods with the pipeline. They are all based on the combined input/output operation model, which can easily complete data processing tasks.

[Learning address]: FFmpeg/WebRTC/RTMP/NDK/Android audio and video streaming media advanced development

[Article Benefits]: Receive more audio and video learning packages, Dachang interview questions, technical videos and learning roadmaps for free. The materials include (C/C++, Linux, FFmpeg webRTC rtmp hls rtsp ffplay srs, etc.) Click 1079654574 to join the group to receive it~

6.1 The difference between -vf and -filter_complex

In FFmpeg, -vfand -filter_complexare used to specify the filter applied during image/video processing, but there are some differences between them.

First of all, -vfit is the abbreviation of simple filter chain, which can only connect multiple simple and linear video filters in series. These filters are executed sequentially, and each filter has only one input and one output. For example:

ffmpeg -i input.mp4 -vf scale=1280:720,rotate=90,crop=1200:600:40:20 output.mp4

The options in the above command -vfconnect three simple filters in series, namely scale (zoom), rotate (rotate) and crop (crop). These filters can only perform simple linear operations during processing, and cannot implement more complex ones. operational needs.

It -filter_complexis more powerful and can support complex, non-linear filter graph structures. -filter_complexThe various filters in the options can be connected and interacted with each other, and complex operations such as image/video synthesis, splicing, and audio mixing can be realized. For example:

ffmpeg -i input1.mp4 -i input2.mp4 -filter_complex "[0:v]scale=640:360[v0];[1:v]scale=1280:720[v1];[v0][v1]overlay=40:20" output.mp4

The options in the above command -filter_complexbuild a complex filter graph structure, respectively scale the two input videos, and use the overlay filter to stitch the two videos together. In this process, various filters can interact and connect at will, thus realizing complex data processing requirements.

In short, -vfit is suitable for simple, linear filter operation scenarios, while it -filter_complexis more powerful and flexible, suitable for complex, non-linear filter operation scenarios.

6.2 filterchain and filtergraph in ffmpeg filter

In ffmpeg filters, filterchain and filtergraph both refer to different ways of processing multiple filters.

• Filterchain is a simple way to arrange filters linearly. It uses commas to concatenate multiple filters, for example: [in] filter1, filter2, filter3 [out]. An image or video frame starts at the input (in), passes through each filter in turn, and finally outputs to the output (out). Filterchain cannot complete complex topological structures, and cannot realize cross-connection between multiple input streams and multiple output streams.

• Filtergraph is more flexible and powerful, and can implement various filter topologies. It uses a graphical way to describe the relationship between all filters, which is very effective for complex media information processing tasks. In the filtergraph, different filters separated by semicolons are called "points", and the processing relationship between them is represented by connecting lines between these points. A filtergraph contains one or more input ports and one or more output ports. Any input can be sent to any filter for processing, and the processed result can also be sent to any output port.

In general, filterchain and filtergraph are both ways to connect and combine multiple filters in ffmpeg filter syntax, while filtergraph can handle more complex topological structures and realize cross-connection between multiple input streams and multiple output streams , and can describe the relationship between all filters in a more intuitive and flexible way.

7. ffmpeg streaming and command parameter order

In FFmpeg, the output of one command can be used as the input of another command through a pipe, so as to realize seamless processing of audio and video data.

In the FFmpeg command, the parameters are divided into global options, input file options and output file options. Global options are valid for all input/output files; input file options only affect specific input files; output file options only affect specific output files.

In general, the global options should be placed first, followed by the input file options, then the input file -iparameters , and finally the output file options and the output file name.

7.1 The difference between the -ss parameter and the -i parameter before and after the -i parameter

Reference Seeking – FFmpeg

When -ssthe parameter is placed -iin front of , FFmpeg will try to jump to the specified position before the input file has been decoded by the decoder, which may cause precision errors, but the processing speed is faster.

On the contrary, when -ssput -iafter , FFmpeg will first decode the input file, and then jump to the specified timestamp, this way can improve the precision and accuracy of the jump timestamp, because FFmpeg already knows the whole video at this time The frame rate, key frame and other information, but the processing time will become longer.

Take generating video thumbnails as an example.

8. The difference between ffmpeg and ffprobe

FFmpeg and ffprobe are two separate tools provided by the FFmpeg multimedia framework for processing and analyzing media files, respectively. Here is a brief difference between the two tools:

• FFmpeg is mainly used for audio and video processing. It can perform many operations such as converting formats, encoding, decoding, clipping, filtering, copying and many more. With FFmpeg, you can write custom scripts for various audio and video processing tasks.

• ffprobe is an analysis tool for examining metadata and technical details of media files. It can provide detailed information about video, audio and subtitle streams, such as codec, resolution, frame rate, bit rate, sample rate and duration, etc. You can use ffprobe to inspect media file properties and compliance, and to build custom media workflows.

Therefore, the application scenarios of FFmpeg and ffprobe are different. If you need to edit, convert or process audio or video files, you can use FFmpeg. If you need to inspect the details or metadata of a media file (such as container format, codec, timebase, etc.), you need to use ffprobe.

Original Link: ffmpeg Getting Started and Questions - Nuggets