FFmpeg learning (1): Introduction to FFmpeg

1. Introduction to FFmpeg

FFmpeg is a set of open source computer programs that can be used to record, convert digital audio and video, and convert them into streams. Adopt LGPL or GPL license. It provides a complete solution for recording, converting, and streaming audio and video. It contains a very advanced audio/video codec library.

Two, FFmpeg composition

• libavformat: used for the generation and analysis of various audio and video packaging formats , including functions such as obtaining information required for decoding to generate decoding context structure and reading audio and video frames;
• libavcodec: used for various types of sound/image encoding and decoding;
• libavutil: Contains some public utility functions;
• libswscale: used for video scene scaling and color mapping conversion;
• libpostproc: used for post-effect processing;
• ffmpeg: a tool provided by this project, which can be used for format conversion, decoding or instant encoding of TV cards, etc.;
• ffsever: an HTTP multimedia instant broadcast streaming server;
• ffplay: It is a simple player that uses ffmpeg library to parse and decode, and display through SDL;

Three, FFmpeg includes class library description

2.1 Class library description

• libavformat-Used for the generation and analysis of various audio and video packaging formats, including functions such as obtaining information required for decoding and reading audio and video data. Various streaming media protocol codes (such as rtmpproto.c, etc.) and audio and video format (de)multiplexing codes (such as flvdec.c, flvenc.c, etc.) are located in this directory.
• libavcodec-Codec for various formats of audio and video. The encoding and decoding codes of various formats (such as aacenc.c, aacdec.c, etc.) are located in this directory.
• libavutil-Contains some public utility functions, including arithmetic operations, character operations, etc.
• libswscale-Provides the functions of scaling, color mapping conversion, image color space or format conversion of the original video.
• libswresample-Provides audio resampling, sampling format conversion and mixing functions.
• libavfilter-Various audio and video filters.
• libpostproc-used for post-effect processing, such as image deblocking, etc.
• libavdevice-used for hardware audio and video capture, acceleration and display.

If you have no previous experience in reading FFmpeg code, it is recommended to read the codes under libavformat, libavcodec and libavutil first. They provide the most basic functions for audio and video development and have the widest application range.

2.2 Common structure

The most commonly used data structures in FFmpeg can be roughly divided into the following categories according to their functions (the following code lines are subject to branch: origin/release/3.4):

1. Package format

• AVFormatContext-describes the composition and basic information of the media file. It is the basic structure that governs the overall situation. It is used as a parameter in many functions throughout the program;
• AVInputFormat-Demultiplexer object, each encapsulation format used as input (such as FLV, MP4, TS, etc.) corresponds to this structure, such as ff_flv_demuxer in libavformat/flvdec.c;
• AVOutputFormat-multiplexer object, each packaging format used as output (such as FLV, MP4, TS, etc.) corresponds to this structure, such as ff_flv_muxer in libavformat/flvenc.c;
• AVStream-used to describe the related data information of a video/audio stream.

2. Codec

• AVCodecContext-describes the data structure of the codec context, which contains the parameter information required by many codecs;
• AVCodec-codec object, each codec format (such as H.264, AAC, etc.) corresponds to a structure, such as ff_aac_decoder in libavcodec/aacdec.c. Each AVCodecContext contains an AVCodec;
• AVCodecParameters-Codec parameters. Each AVStream contains an AVCodecParameters, which is used to store the codec parameters of the current stream.

3. Network Protocol

• AVIOContext-a structure that manages input and output data;
• URLProtocol-describes the protocol used for audio and video data transmission. Each transmission protocol (such as HTTP, RTMP), etc., corresponds to a URLProtocol structure, such as ff_http_protocol in libavformat/http.c;
• URLContext-encapsulates the protocol object and protocol operation object.

4. Data storage

• AVPacket-store compressed data after encoding and before decoding, that is, ES data;
• AVFrame-store the original data before and after decoding, such as video data in YUV format or audio data in PCM format, etc.;