This article is a rewrite... I wrote about 2,000 words a few days ago. I wanted to wait for the series to be released together, so I saved the draft. I flipped through it occasionally today and found only a few words on the draft...all missing. At the time, I was in a broken mood.
Rewriting this article with gritted teeth can also be regarded as laying a firmer foundation for the interpretation of ffmpeg.
Introduction
ffmpeg has changed a lot in recent versions. Here are two pictures circulating on the Internet, which can be regarded as the first knowledge of ffmpeg.
ffmpeg decoder flow chart
ffmpeg encoder flow chart
The picture is relatively large, please download and view it.
Pink background functions: API functions of FFmpeg.
Functions with white background: Internal functions of FFmpeg.
Functions with yellow background: The functions in the URLProtocol structure include the functions of reading and writing various protocols.
Functions with green background: The functions in the AVOutputFormat structure include the functions of reading and writing various package formats (AVI, MKV, FLV.).
Functions with blue background: functions in the AVCodec structure, including the functions of encoding and decoding.
I think the core point of the ffmpeg structure is the template mode, and the template mode is implemented in the c language (the core skill is the function pointer).
The most obvious manifestation is the method of the white background box to the left of the dotted box.
To give a few examples: The URLProtocol structure contains the following protocol handler pointers:
url_open()：打开
url_read()：读取
url_write()：写入
url_seek()：调整进度
url_close()：关闭
Different protocols have different implementation functions corresponding to the above interfaces. The URLProtocol structure ff_file_protocol corresponding to the
File protocol
(ie file): url_open() -> file_open() -> open()
url_read() -> file_read() -> read ()
url_write() -> file_write() -> write()
url_seek() -> file_seek() -> lseek()
url_close() -> file_close() -> close()
We will accumulate more details in the next part.
preface
The system I use is os x, assuming we use the linux series of systems. This will affect some environment configurations. When compiling ffmpeg, use config for configuration. The macros in config.h generated after compilation in different environments will have different configurations.
thread
Since C language does not have a unified implementation of thread, different platforms will have different implementations. In addition, ffmpeg also implements a thread itself.
In the thread.h header file, ffmpeg does some configuration for threads.
In a Linux environment
#if HAVE_PTHREADS
#include <pthread.h>
#define AVMutex pthread_mutex_t
#define AVOnce pthread_once_t
#define ff_thread_once(control, routine) pthread_once(control, routine)This is a very commonly used configuration item.
For example, ff_thread_once is actually pthread_once in the Linux environment. A brief introduction to pthread_once
int pthread_once(pthread_once_t *once_control, void (*init_routine) (void))；
Function: This function uses the once_control variable whose initial value is PTHREAD_ONCE_INIT to ensure that the init_routine() function is executed only once in the execution sequence of this process. In a multi-threaded programming environment, although the pthread_once() call will appear in multiple threads, the init_routine() function is executed only once, and it is uncertain which thread is executed in and is determined by the kernel scheduling.
In fact, this function does not care who the running function pointer is, whether this function is called for the first time, it only cares whether the current state value of the pthread_once_t variable has been executed or not executed.
avcodec_register_all
There are many methods of register_all in ffmpeg, which are generally used to register various components, such as registering decoders, registering encoders, registering mixers, etc.
And avcodec_register_all is the function used to register all decoders and encoders.
void avcodec_register_all(void)
{
ff_thread_once(&av_codec_next_init, av_codec_init_next);
}
static void av_codec_init_next(void)
{
AVCodec *prev = NULL, *p;
void *i = 0;
while ((p = (AVCodec*)av_codec_iterate(&i))) {
if (prev)
prev->next = p;
prev = p;
}
}
const AVCodec *av_codec_iterate(void **opaque)
{
// i 最初的值是0， 表示指向0地址的指针
uintptr_t i = (uintptr_t)*opaque;
//使用地址值 0 来当做index。
const AVCodec *c = codec_list[i];
ff_thread_once(&av_codec_static_init, av_codec_init_static);
if (c)//地址值+1
*opaque = (void*)(i + 1);
return c;
}
static AVOnce av_codec_static_init = AV_ONCE_INIT;
static void av_codec_init_static(void)
{
for (int i = 0; codec_list[i]; i++) {
if (codec_list[i]->init_static_data)
codec_list[i]->init_static_data((AVCodec*)codec_list[i]);
}
}For the sake of reading, I have reversed the order of function definitions. In c, from top to bottom, if there is no definition or declaration, it will report an error if the function is used directly.
First of all, avcodec_register_all actually uses ff_pthread_once to ensure that it will only be registered once (this is an improved method in recent versions, the early ffmpeg did not use ff_pthread_once). The method actually defined is av_codec_init_next.
An AVCodec structure represents a decoder or encoder. We will explain this structure later. As for the rank, this level of cognition is enough to understand the code. There is a next pointer in the AVCodec structure, yes, this is a typical linked list structure. The av_codec_init_next method is to initialize this linked list and string all AVCodec structure variables into a linked list.
The role of av_codec_iterate is to return an AVCodec object in turn. Reading this code for the first time may be a little confusing, such as using a pointer to a pointer or something. The core idea here is to use the address value of the pointer instead of an int type value to traverse the codec_list list. It may not be very accurate, but doing so does save the memory address of an iterator... We don't need to actually open up an address of type int, store the index, and then use a pointer to point to the index to ensure that the Different methods can also make this value +1.
But this is not important. In short, av_codec_iterate traverses the codec_list and returns all AVCodecs in turn. Also called av_codec_init_static (only run sequentially)
The role of av_codec_init_static is to sequentially call the methods pointed to by the init_static_data pointers in all codec_lists. Used to initialize this decoder or encoder.
Although most decoders and encoders do not implement this method. But as the designer is there any room for improvement in this code? For example, when using this structure for the first time, call the initialization method instead of calling the methods of all decoders at once, because not all types of decoders will be used in actual use. Please explain the reason for this design.
av_register_all
The former king! In the past, whenever ffmpeg was used, av_register_all would be called first, just like the name, it was used to register everything. However, during the evolution of ffmpeg, the functional components have been weakened.
muxer: packager, used to package audio and video into avi mp4 and other formats
demuxer: Decapsulator, restores avi mp4 and other videos to video and audio sources
void av_register_all(void)
{
ff_thread_once(&av_format_next_init, av_format_init_next);
}
static void av_format_init_next(void)
{
AVOutputFormat *prevout = NULL, *out;
AVInputFormat *previn = NULL, *in;
ff_mutex_lock(&avpriv_register_devices_mutex);
//将所有 muxer（封装器，用来将音视频封装成avi mp4等格式）组成链表
for (int i = 0; (out = (AVOutputFormat*)muxer_list[i]); i++) {
if (prevout)
prevout->next = out;
prevout = out;
}
if (outdev_list) {
for (int i = 0; (out = (AVOutputFormat*)outdev_list[i]); i++) {
if (prevout)
prevout->next = out;
prevout = out;
}
}
//将所有demuxer组成链表
for (int i = 0; (in = (AVInputFormat*)demuxer_list[i]); i++) {
if (previn)
previn->next = in;
previn = in;
}
if (indev_list) {
for (int i = 0; (in = (AVInputFormat*)indev_list[i]); i++) {
if (previn)
previn->next = in;
previn = in;
}
}
ff_mutex_unlock(&avpriv_register_devices_mutex);
}The actual code has just been used to register the muxer and demuxer. There are two quantities here, one is outdev_list and one is indev_list. These two objects can be set by developers themselves (avpriv_register_devices) to customize which muxers and demuxers are needed.
avfilter_register_all
Used to concatenate all filters into a linked list. Filter is a food processing component given by ffmpeg, which can perform various operations on video, such as scaling, cropping, and watermarking.
static void av_filter_init_next(void)
{
AVFilter *prev = NULL, *p;
void *i = 0;
while ((p = (AVFilter*)av_filter_iterate(&i))) {
if (prev)
prev->next = p;
prev = p;
}
}
void avfilter_register_all(void)
{
ff_thread_once(&av_filter_next_init, av_filter_init_next);
}
avformat_network_init
Used to initialize the network.
int avformat_network_init(void)
{
#if CONFIG_NETWORK
int ret;
//初始化网络(windows平台下生效)
if ((ret = ff_network_init()) < 0)
return ret;
//初始化tls
if ((ret = ff_tls_init()) < 0)
return ret;
#endif
return 0;
}
ffurl_register_protocol
It was originally used to register the URLProtocol protocol list, but this method cannot be found in the current ffmpeg, so there is no need to register the URLProtocol protocol list.
register_exit
This is a pretty useless method. Defined in cmdutils.c
static void (*program_exit)(int ret);
void register_exit(void (*cb)(int ret))
{
program_exit = cb;
}
void exit_program(int ret)
{
if (program_exit)
program_exit(ret);
exit(ret);
}It registers a function pointer, which is equivalent to a callback, which is processed in exit_program and used to do some aftermath work, such as releasing resources.
But the problem lies in the method exit_program. After calling the cleanup method, it will continue to call exit to exit the current process. That's why it's hard to use it in real projects. Most of the time you don't intend to just exit the process when you end the ffmpeg operation.