[Introduction] FFmpeg is a complete cross-platform audio and video solution, which can be used for audio and video transcoding, transpackaging, retweeting, recording, streaming processing and other application scenarios. FFmpeg enjoys a high reputation in the audio and video field and is known as the Swiss Army Knife of the audio and video industry. At the same time, FFmpeg has three major weapons that we should be clear about. They are the three Musketeers of ffmpeg, ffprobe and ffplay.
text
The Three Musketeers ffmpeg, ffprobe, and ffplay all have their own usage scenarios and special skills. In the process of audio and video development, if these three tools can be used reasonably, there will be more results with less effort. Next, we will introduce the use skills of the three of them in combination with actual cases.
ffmpeg
ffmpeg should be the core tool in the FFmpeg toolset, supporting a variety of encoders, decoders, encapsulation formats, and filter functions. In the actual development process, we can directly use the ffmpeg executable program, or reference dependencies in the form of dynamic libraries or static libraries. However, in technical research, executable programs are mostly used because the operation is simpler and the cost is lower.
In fact, the ffmpeg version we use can be customized. Most companies have their own ffmpeg private library version, and the official ffmpeg version has been tailored and customized according to the company's business. Because many companies' products are a closed loop, especially player solutions, do not need to support all encoding formats, decoding formats and encapsulation formats.
For example, our company's cloud-on-demand service transcodes all uploaded video files and audio files into standard video H264 and audio AAC encoding formats, and finally stores them in m3u8 format. The player SDK we provide only needs to support the decoding of H264 and AAC encoding formats, and does not need to care about the support for other audio and video encoding formats. Such a player solution is very lightweight. At the same time, it is also compatible with most general-purpose players on the market.
Next, we will introduce how to determine which encoders, decoders, encapsulation formats and filter functions are supported by the current ffmpeg version.
Encoder
If you want to see which encoders are supported by the current ffmpeg version, you can use the following command:
ffmpeg -encoders
Example of running results: (partial content)
Encoders:
V..... = Video
A..... = Audio
S..... = Subtitle
.F.... = Frame-level multithreading
..S... = Slice-level multithreading
...X.. = Codec is experimental
....B. = Supports draw_horiz_band
.....D = Supports direct rendering method 1
------
V..... flv FLV / Sorenson Spark / Sorenson H.263 (Flash Video) (codec flv1)
V..... gif GIF (Graphics Interchange Format)
V..... h263 H.263 / H.263-1996
V.S... mpeg2video MPEG-2 video
V.S... mpeg4 MPEG-4 part 2
Under what circumstances would we use this command? For example, when you are using ffmpeg encoding, the result is wrong, you first need to use this command to check whether the current ffmpeg supports the target encoder. Here you may ask, is there no unified version of ffmpeg? Why are the supported encoders different? The official version of ffmpeg on the official website is definitely unified, but in actual projects, we cannot use all the encoders, decoders, filters and other components, and they will generally be cropped.
There may be another reason for cutting ffmpeg, which is limited by the size of the client installation package. The size of the full ffmpeg is tens of megabytes, which may account for half of the entire installation package, so it is often necessary to perform ffmpeg Cropped.
Audio and video development document video data + mind map skill point route!
decoder
If you want to see which codecs are supported by the current ffmpeg version, you can use the following command:
ffmpeg -decoders
The output result will initially display the version numbers of different components of the current ffmpeg version. In fact, ffmpeg itself is a collection of tools that contains different functional modules. Among them, including libavutil, libavcodec, libavformat, libavdevice, libavfilter, libswscale, libswresample seven parts, together constitute a powerful ffmpeg tool.
Example of running results: (partial content)
Decoders:
V..... = Video
A..... = Audio
S..... = Subtitle
.F.... = Frame-level multithreading
..S... = Slice-level multithreading
...X.. = Codec is experimental
....B. = Supports draw_horiz_band
.....D = Supports direct rendering method 1
------
V...BD flv FLV / Sorenson Spark / Sorenson H.263 (Flash Video) (codec flv1)
V....D gif GIF (Graphics Interchange Format)
V....D h261 H.261
V...BD h263 H.263 / H.263-1996, H.263+ / H.263-1998 / H.263 version 2
VFS..D h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
V..... h264_vda H.264 (VDA acceleration) (codec h264)
VFS..D hevc HEVC (High Efficiency Video Coding)
V.S.BD mpeg1video MPEG-1 video
V.S.BD mpeg2video MPEG-2 video
V.S.BD mpegvideo MPEG-1 video (codec mpeg2video)
VF..BD mpeg4 MPEG-4 part 2
If you roughly browse the list of encoders and decoders, you will feel that ffmpeg supports more decoders than encoders? Yes, your feeling is not wrong. ffmpeg itself is more often used for decoding. The encoder is a video encoding standard defined by some video manufacturers or third-party organizations, and then the ffmpeg team decides whether to support the corresponding decoding function as needed. For example, when encoding H264 video, More use is openH264 or x264 encoding library.
In addition, it should be noted that there is a similar equation:
ffmpeg -codecs = （ffmpeg -encoders）+（ffmpeg -decoders）。
This means that you can also use the ffmpeg -codecscommand to view the overall encoder and decoder conditions.
Package format
When it comes to the package format, it is easier for everyone to understand. In fact, it is the suffix of the video files we usually see, such as mp4, flv, mkv, webm, ts, 3gp, etc. ffmpeg needs to use the encapsulation format when encoding, such as encapsulating AAC audio and H264 video into mp4 files; when decoding, it also needs to use decapsulation formats, such as decapsulating mp4 files into AAC audio and H264 video . We can use the following commands to view the encapsulation and decapsulation formats supported by ffmpeg:
ffmpeg -formats
Example of running results: (partial content)
File formats:
D. = Demuxing supported
.E = Muxing supported
--
E 3gp 3GP (3GPP file format)
D aac raw ADTS AAC (Advanced Audio Coding)
DE ac3 raw AC-3
D ape Monkey's Audio
DE avi AVI (Audio Video Interleaved)
DE flv FLV (Flash Video)
DE g722 raw G.722
gif GIF Animation
DE h263 raw H.263
DE h264 raw H.264 video
filter
The filter function is generally only used when performing some more complex operations, such as adding textures to videos, mixing streams and merging.
You can use the following command to see which filters are supported by the current ffmpeg version:
ffmpeg -filters
Example of running results: (partial content)
Filters:
T.. = Timeline support
.S. = Slice threading
..C = Command support
A = Audio input/output
V = Video input/output
N = Dynamic number and/or type of input/output
| = Source or sink filter
... acompressor A->A Audio compressor.
... acrossfade AA->A Cross fade two input audio streams.
Filters support a variety of types, including timeline, thread slice, command line, control audio input and output, control video input and output, dynamically control the number and type of input and output, source filter, and different functions can choose the corresponding type of filter .
ffprobe
ffprobe in FFmpeg Three Musketeers is a tool for viewing multimedia information of files. It is super simple to use and the output information is very detailed. In the process of audio and video development, when we need to process a multimedia file, we first need to know its basic information, such as is it an audio file or a video file? In the case of video files, does it contain an audio stream? And the corresponding audio and video parameter information, bit rate, resolution, sampling rate, number of channels, sampling bits, frame rate, color space, etc.
View audio information
Using ffprobe to view audio information is also very simple. You can specify many parameters. The simplest way to use it can refer to the following commands:
ffprobe demo.ape
Example of running result:
Input #0, ape, from 'demo.ape':
Metadata:
TITLE : Suffering
Copyright : QQ Music
ALBUM : Thank you lover
ARTIST : [www.51ape.com] Jiawei Li
Duration: 00:04:22.03, start: 0.000000, bitrate: 793 kb/s
Stream #0:0: Audio: ape (APE / 0x20455041), 44100 Hz, stereo, s16p
Is it very simple? Let's briefly analyze the above output information, this is a music file, the title is "suffering", the music duration is 4 minutes and 22 seconds, the bit rate is 793kb/s, the audio format is ape, the sampling rate is 44100 Hz, stereo, sampling The format is s16p.
View video information
Using ffprobe to view video information is also very simple, you can refer to the following commands:
ffprobe yingzi.mp4
Example of running result:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'yingzi.mp4':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: mp41mp42isom
creation_time : 2021-02-07 12:48:08
Duration: 00:00:15.05, start: 0.000000, bitrate: 825 kb/s
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 105 kb/s (default)
Metadata:
creation_time : 2021-02-07 12:48:08
handler_name : Core Media Audio
Stream #0:1(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, smpte170m/smpte170m/bt709), 320x568, 713 kb/s, 29.97 fps, 29.97 tbr, 600 tbn, 1200 tbc (default)
Metadata:
creation_time : 2021-02-07 12:48:08
handler_name : Core Media Video
Through the above output information, we can know that this video contains both audio stream and video stream, video duration is 15 seconds, audio encoding AAC, video encoding H264 (Main), color space is yuv420p, resolution is 320x568, frame rate is 29.97.
Learning materials collection address: https://docs.qq.com/doc/DQm1VTHBlQmdmTlN2
ffplay
ffplay is a multi-function player in the FFmpeg Three Musketeers. ffplay can play both audio and video. It is one of the basic tools commonly used by staff engaged in audio and video related development. It is very necessary to master the use of ffpaly. Next, let's take a look at how to use ffplay.
play audio file
Playing a piece of music with ffplay is very simple. For details, refer to the following commands:
ffplay demo.ape
While playing, an audio spectrogram will appear by default, as shown in the following figure:
There are two display modes for playing graphics, one is waveform chart, showmode=1, the other is spectrogram, showmode=2, the default music playback is spectrogram. If you want to display the waveform graph, you can use the following command:
ffplay -showmode 1 demo.ape
The graph is displayed as follows:
If you don't want to display any playback graphics, it is also possible, set showmode=0, and run the following command:
ffplay -showmode 0 demo.ape
This will only have the information output as shown in the figure below.
Play video files
It is also very simple to use ffplay to play a video. For details, refer to the following commands:
ffplay bunny.mp4
The playback effect is as follows:
Not only that, when the video file has multiple audio or multiple video, ffplay can also specify which audio or video to play. By default, the audio and video with index=0 are played. For example, specify to play the second audio:
ffplay test.mp4 -ast 1
For example, specify to play the second video:
ffplay test.mp4 -vst 1
In addition, everyone knows that when audio and video exist at the same time, there is a problem of audio and video synchronization during playback, so how does ffplay handle it? In fact, ffplay is based on the audio timestamp by default. Of course, we can also specify the video or system timestamp as the benchmark. Commands based on video timestamps for playback:
ffplay test.mp4 -sync video
The command to play based on the system timestamp:
ffplay test.mp4 -sync ext
Just now, we already know that ffplay can play audio raw data PCM format files, so can ffplay play video raw data YUV format video files? The answer is also possible, the command is as follows:
ffplay test.yuv -f rawvideo -pixel_format yuv420p -s 544*960
end
Today, I briefly introduced the three great gods in the FFmpeg toolset - ffmpeg, ffprobe, and ffplay. Skilled use of them will greatly help your audio and video development work. In fact, we can use them reasonably not only in work, but also in life. They, such as DIY some video effects that Douyin and Kuaishou cannot do, etc. In fact, there are many more in-depth ways of using them. It is impossible for an article to introduce them in a comprehensive way. Interested friends can try it out by themselves, or they can comment and leave a message to exchange and learn. I have been engaged in audio and video work for many years, and I have many tools and working skills that I hope to share with you, come on!