ffmpeg study notes (three)

1. FFmpeg decoding API analysis

1. The avformat_open_input analysis
   function avformat_open_input will determine the file format according to the provided file path. In fact, it is through this step to determine which Demuxer to use.

For example: if it is flv, Demuxer will use the corresponding ff_flv_demuxer, so the corresponding key life cycle methods read_header, read_packet, read_seek, and read_close will all use the function specified by the function pointer in the flv's Demuxer. The read_header will construct the AVStream structure to facilitate the subsequent steps to continue to use AVStream as an input parameter.

2. avformat_find_stream_info analysis
   The function of this method is to fill all the MetaData information of the Stream. The method will first find the right decoder, then open the corresponding decoder, and then use the read_packet function in Demuxer to read a piece of data for decoding. Of course, the more data decoded, the more accurate the analyzed stream information. If it is a local resource, then accurate information can be obtained soon. But for network resources, it will be slower, so this function has several parameters to control the length of the read data, one is probe size, one is max_analyze_duration, and the other is fps_probe_size. These three parameters jointly control the decoded data If the value of the configured parameters is smaller, the execution time of this function will be faster, but the information in the AVStream structure (video width, height, fps, encoding type) will be inaccurate.

3. av_read_frame analyzes
   that the data read by this method is AVPacket. The function developed for developers in the early version of FFmpeg is actually av_read_packet, but the developer needs to deal with the situation that the data in AVPacket cannot be processed by the decoder. The problem of caching unprocessed compressed data. So in the new version of FFmpeg, this function is provided to handle this situation. The implementation of this function will first be delegated to Demuxer's read_packet method. Of course, read_packet will return the data here after the processing of the serving layer and the protocol layer, and data buffering will be performed in this function.

For audio streams, one AVPacket may contain multiple AVFrames, but for one video stream, one AVPacket only contains one AVFrame, and this function will only return one AVPacket structure in the end.

4. avcodec_decode analysis
   The method contains two parts: one is to decode the video, and the other is to decode the audio. In the above function analysis, we know that the decoding will be entrusted to the corresponding decoder to implement. When the decoder is turned on, the corresponding decoder implementation is found. For example, for decoding H264, ff_h264_decoder will be found. Among them, there will be the implementation of the corresponding life cycle functions. The most important ones are the init, decode, and close methods, which correspond to the operations of opening the decoder, decoding and closing the decoder, and the decoding process is to call the decode method.

5. avformat_close_input analysis
   This function is responsible for releasing the corresponding resources. First, it will call the lifecycle read_close method in the corresponding Demuxer, then release the AVFormatContext, and finally close the file or remote network link.

2. FFmpeg encoding API analysis

1. avformat_alloc_output_context2 Analysis
   This function needs to call the method avformat_alloc_context to allocate an AVFormatContext structure. Of course, the most important thing is to find the corresponding format according to the Muxer and Demuxer parts registered in the previous step (that is, the package format part). It may be flv format, MP4 format, mov format, or even MP3 format, etc. If the corresponding format is not found (it should be because the switch of this format is not turned on in the configure option), then it will return to find the right Format error prompt. When calling the API, you can use av_err2str to convert the returned integer error code into a human-readable string, which is a very useful tool function in debugging. This function will finally assign the found format to oformat of type AVFormatContext.

2. The avio_open2 analysis
   first calls the function ffurl_open to construct the URLContext structure, which contains URLProtocol (you need to go to the protocol linked list registered in the first step register_protocol) to find; then it will call the avio_alloc_contex method to allocate the AVIOContext structure , And pass in the URLProtocol constructed in the previous step; then assign the AVIOContext structure assigned in the previous step to the AVFormatContext property.

The following is the framework diagram between the structures summarized in the above description. You can refer to this diagram for further understanding:

The process of avio_open2 also happens to be an inverse process of the above analysis of the avformat_open_input process. The encoding process and decoding process are logically an inverse process, so in the process of FFmpeg implementation, they are also inverse processes to each other.

3. Encoding other API (step) analysis
   The other steps of encoding are also an inverse process of decoding. The avformat_find_stream_info in the decoding process corresponds to the encoding avformat_new_stream and avformat_write_header.

The avformat_new_stream function will fill in the information of the audio stream or video stream, allocate the AVStream structure, distribute the channel, sampling rate, representation format, encoder and other information in the audio stream, and distribute the width, height, frame rate, Represents information such as format and encoder.
The avformat_write_header function and the read_header in the decoding process happen to be an inverse process, so I won’t go into details here.
The next step is the coding phase:

1. Take the manually encapsulated AVFrame structure as the input of the avcodec_encodec_video method, then encode it into an AVPacket, and then call the av_write_frame method to output to the media file.

2. The av_write_frame method will use the encoded AVPacket structure as the input of the write_packet life cycle method in Muxer. The write_packet will add the header information of its own encapsulation format, and then call the protocol layer and write it to a local file or network server.

3. The last step is av_write_trailer (this function has a very big pit, if the write_header operation is not executed, the write_trailer operation is executed directly, the program will be Carsh off directly, so these two functions must appear in pairs), av_write_trailer will put the AVPacket that has no output All are thrown to the protocol layer for output, and then Muxer's write_trailer life cycle method will be called (different formats, and the written tails are different).