FFmpeg not only provides three executable programs of ffmpeg, ffplay and ffprobe, but also provides eight tool libraries, enabling developers to call the functions in the library, so as to achieve more precise customized development requirements. The names of these eight libraries are avcodec, avdevice, avfilter, avformat, avutil, postproc, swresample, swscale, and these libraries are introduced below.
1. avcodec
Avcodec is FFmpeg's audio and video codec library, which includes various audio encoding libraries and decoding libraries, as well as various video encoding libraries and decoding libraries. Through avcodec, the original audio and video data can be encoded into a data compression package conforming to a certain code stream rule, and the data compression package can also be decompressed into the original audio and video data according to the specified code stream rule. Although avcodec has built-in most audio and video codec libraries, some streams need to integrate third-party codec libraries. For example, the video format h264 requires the integration of third-party x264, the video format h265 requires the integration of third-party x265, and the audio format mp3 It is required to integrate third-party mp3lame, etc. Many lib***.c codes in the libavcodec directory are used to integrate third-party codec libraries.
2. avdevice
Avdevice is the audio and video device library of FFmpeg. It contains various input and output device libraries of audio and video. The input device refers to the device that collects audio and video signals, and the output device refers to the device that renders audio and video images. Of course, FFmpeg will not directly operate the device hardware, but realize it through third-party software packages. For example, the VFW capturer of the Windows platform (VFW’s full name is Video for Windows) and the upgraded version of VFW’s DirectShow capturer are used to capture media signals; rendering The media screen uses the GDI receiver of the Windows platform (the full name of GDI is Graphics Device Interface), and the cross-platform SDL2 media development library (the full name of SDL is Simple DirectMedia Layer). Of course, FFmpeg also supports the sound processing library OpenAL (full name Open Audio Library) and graphics processing library OpenGL (full name Open Graphics Library).
3. avfilter
avfilter is the audio and video filter library of FFmpeg, which contains various filter packages for processing and editing audio and video, among which the source code file name of the audio filter is like af_***.c, and the source code of the video filter The file name is in the form of vf_***.c. Audio filters are mostly used for processing such as adjusting parameters, mixing audio, etc., and video filters are mostly used for processing such as transforming video, special effect images, and adding components.
4. avformat
Avformat is the audio and video format library of FFmpeg, which includes various media file format libraries and various network communication protocol libraries. The format library includes not only video formats mp4, avi, mov, rm, etc., audio formats mp3, wav, aac, pcm, etc., but also image formats such as jpeg, gif, png, yuv, etc. The protocol library not only includes the file protocol file, conventional communication protocols http, ftp, tcp, udp, etc., but also includes streaming media transmission protocols such as rtsp, rtmp, hls, srt, etc.
5. avutil
avutil is the audio and video tool library of FFmpeg. It contains common general tools and various algorithm libraries. The general tools include dictionary operations, logging, cache interaction, thread processing, and encryption and decryption libraries aes, md5, sha , base64, etc.; various algorithms include queuing algorithm fifo, sorting algorithm qsort, hash table hash, binary tree tree, etc. In addition, avutil also includes public functions such as color space and audio sampling.
6. postproc
postproc is FFmpeg's audio and video post-effect processing library. It is mainly used for post-effect processing. If filters are used in the code, this library must be linked when compiling, because the filters use some basic functions of postproc .
7. swresample
swresample is the audio resampling library of FFmpeg. It is mainly used for audio resampling related functions, such as changing audio from mono to multi-channel, changing audio sampling frequency, converting audio data format, and so on.
8. swscale
swscale is FFmpeg's video image conversion library, which is mainly used for image scaling, color space conversion and other functions. Color space conversion is sometimes called pixel format conversion, such as converting video frames from YUV format to RGB format.