Audio and video development - ffmpeg introduction - series two

Table of contents

1. FFmpeg core structure

2. Decoding process 

3. Implementation of FFmpeg decoding 

Fourth, FFmpeg encoding implementation

 Five, FFmpeg transcoding implementation 

---

1. FFmpeg core structure

AVFormatContext: structure of decapsulation function, including file name, audio and video stream, duration, bit rate and other information; AVCodecContext:
codec context, a structure that must be used for encoding and decoding, including codec type, video width Information such as high, number of audio channels and sampling rate;
AVCodec: a structure storing codec information;
AVStream: a structure storing audio or video stream information;
AVPacket: storing audio or video encoding data;
AVFrame: storing audio or video decoding data (raw data)

2. Decoding process

 

3. Implementation of FFmpeg decoding 

The decoding implements decoding the video data in the compressed domain into YUV data in the pixel domain. The implementation process can be roughly shown in the following figure

As can be seen from the figure, it can be roughly divided into the following three steps:

1. First there is video in the compressed domain to be decoded.
2. Next, a decoder is obtained according to the compressed format of the compressed domain.
3. The output of the final decoder is the YUV data in the pixel domain

Fourth, FFmpeg encoding implementation

From the figure, we can roughly see the process of video encoding:

1. First there is uncompressed YUV raw data.
2. The second is to choose a specific encoder based on the format you want to encode.
3. The output of the final encoder is the encoded video frame

 Five, FFmpeg transcoding implementation 

Schematic diagram of how a traditional transcoding program works

   

Purpose of encapsulation:

1. It is to simultaneously store video stream (Video Stream), audio stream (Audio Stream), subtitle (Subtitle), attachment (t), data (d) and other content in a file stream (Stream). This is exactly what "multiplexing" means (time-division multiplexing).
2. It is to ensure the reliable and fast transmission of data in the network environment.
Purpose of coding:

It is for compressing media data. Different from the compression of general file data, when compressing images or audio, it can be compressed with the help of image characteristics (such as front and back association, adjacent block association) or sound characteristics (auditory model), which can achieve higher than general compression technology. compression ratio