ijkplayer source code analysis (3): setOption process and common parameter settings

introduction

When using ijkplayer, we can configure some parameters to enable or disable certain functional modules, or choose to use a certain method. For example, whether to use hard decoding or soft decoding through parameter configuration, whether to use AudioTrack or OpenSL for Android audio playback, whether to enable SoundTouch, etc. What parameter configuration does ijkplayer have? What is the role of each parameter? What are the commonly used parameter configurations?

This article is based on the A4ijkplayer project for ijkplayer source code analysis. This project is to change ijkplayer to CMake-based compilation, which can be imported into Android Studio to compile and run, which is convenient for code search, function jump, single-step debugging, call stack trace, etc.

The main content structure of this article:

• 1. Common parameter settings of ijkplayer
• 2. ijkplayer parameter setting process
• 3. The five types of parameter settings of ijkplayer and the functions of each parameter

1. Common parameter settings of ijkplayer

// OPT_CATEGORY_PLAYER 类的参数设置

// 是否开启 Mediacodec 硬解，1 为开启，0 为关闭
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec", 1);
// 关闭 Mediacodec 硬解，使用 FFmpeg 软解
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec", 0);

// 音频播放是否使用 openSL，1：使用 openSL，0：使用 AudioTrack
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "opensles", 1);

// 当 CPU 处理不过来的时候的丢帧帧数，默认为 0，参数范围是 [-1, 120]，详情见：ff_ffplay_options.h
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "framedrop", 5);

// 在资源准备好后是否自动播放，1：自动播放，0：准备好后不自动播放
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "start-on-prepared", 1);

// 使用 mediacodec 时是否根据 meta 信息自动旋转视频
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec-auto-rotate", 1);

// 使用 mediacodec 时是否处理分辨率改变
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec-handle-resolution-change", 0);

// 设置视频显示格式
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "overlay-format", IjkMediaPlayer.SDL_FCC_RV32);

// 是否开启精准 seek，默认关闭
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "enable-accurate-seek", 1)

// 是否启用 soundtouch，配合 setSpeed 实现变速播放
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "soundtouch", 1)
ijkMediaPlayer.setSpeed(0.5f);


// OPT_CATEGORY_CODEC 类的参数设置

// 是否开启跳过 loop filter，0 为开启，画面质量高，但解码开销大，48 为关闭，画面质量稍差，但解码开销小
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_CODEC, "skip_loop_filter", 0);


// OPT_CATEGORY_FORMAT 类的参数设置

// 设置探测输入的分析时长，单位：微秒，详情见：libavformat/options_table.h，通常设置 1 达到首屏秒开效果
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT, "analyzeduration", 1);

// 在每个数据包之后启用 I/O 上下文的刷新
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT, "flush_packets", 1);

2. ijkplayer parameter setting process

First look at the definition of the parameter setting interface in the Java layer IjkMediaPlayer.java:

    public void setOption(int category, String name, String value) {
    
    
        _setOption(category, name, value);
    }

    public void setOption(int category, String name, long value) {
    
    
        _setOption(category, name, value);
    }

    private native void _setOption(int category, String name, String value);
    private native void _setOption(int category, String name, long value);

The above two interfaces are similar, both in the form of category, key, and value . The difference lies in whether the value is of type String or long. It can be seen that it has three parameters:

• Category: The large category of parameters to be set . There are five categories defined in ijkplayer, which will be introduced in detail later:
  • format： FFP_OPT_CATEGORY_FORMAT 1
  • codec： FFP_OPT_CATEGORY_CODEC 2
  • sws： FFP_OPT_CATEGORY_SWS 3
  • player： FFP_OPT_CATEGORY_PLAYER 4
  • swr： FFP_OPT_CATEGORY_SWR 5
• key: the key of the parameter to be set under the category
• value: the value of the parameter to be set under the category

2.1 Save the parameters set by the upper layer

After the Java layer calls _setOption, it will call the method ijkmedia/ijkplayer/android/ijkplayer_jni.cin IjkMediaPlayer_setOption(), as follows:

static void
IjkMediaPlayer_setOption(JNIEnv *env, jobject thiz, jint category, jobject name, jobject value)
{
    
    
    // ...
    c_name = (*env)->GetStringUTFChars(env, name, NULL );
    c_value = (*env)->GetStringUTFChars(env, value, NULL );
    // ...
    ijkmp_set_option(mp, category, c_name, c_value);
    // ...
}

static void
IjkMediaPlayer_setOptionLong(JNIEnv *env, jobject thiz, jint category, jobject name, jlong value)
{
    
    
    // ...
    c_name = (*env)->GetStringUTFChars(env, name, NULL );
    // ...
    ijkmp_set_option_int(mp, category, c_name, value);
}

Then calls to the method ijkmedia/ijkplayer/ijkplayer.cof ijkmp_set_option():

void ijkmp_set_option(IjkMediaPlayer *mp, int opt_category, const char *name, const char *value)
{
    
    
    pthread_mutex_lock(&mp->mutex);
    ffp_set_option(mp->ffplayer, opt_category, name, value);
    pthread_mutex_unlock(&mp->mutex);
}

void ijkmp_set_option_int(IjkMediaPlayer *mp, int opt_category, const char *name, int64_t value)
{
    
    
    pthread_mutex_lock(&mp->mutex);
    ffp_set_option_int(mp->ffplayer, opt_category, name, value);
    pthread_mutex_unlock(&mp->mutex);
}

Then transfer ijkmedia/ijkplayer/ff_ffplay.cto ffp_set_option()the method, actually call in ijkmedia/ijkplayer/ff_ffplay.cto ffp_get_opt_dict()get the dictionary corresponding to the major categories in FFPlayer and then save the parameters according to the classification:

void ffp_set_option(FFPlayer *ffp, int opt_category, const char *name, const char *value)
{
    
    
    AVDictionary **dict = ffp_get_opt_dict(ffp, opt_category);
    av_dict_set(dict, name, value, 0);
}

void ffp_set_option_int(FFPlayer *ffp, int opt_category, const char *name, int64_t value)
{
    
    
    AVDictionary **dict = ffp_get_opt_dict(ffp, opt_category);
    av_dict_set_int(dict, name, value, 0);
}

static AVDictionary **ffp_get_opt_dict(FFPlayer *ffp, int opt_category)
{
    
    
    switch (opt_category) {
    
    
        case FFP_OPT_CATEGORY_FORMAT:   return &ffp->format_opts;
        case FFP_OPT_CATEGORY_CODEC:    return &ffp->codec_opts;
        case FFP_OPT_CATEGORY_SWS:      return &ffp->sws_dict;
        case FFP_OPT_CATEGORY_PLAYER:   return &ffp->player_opts;
        case FFP_OPT_CATEGORY_SWR:      return &ffp->swr_opts;
        default:
            av_log(ffp, AV_LOG_ERROR, "unknown option category %d\n", opt_category);
            return NULL;
    }
}

2.2 Parameters of application settings

setOption()After the execution, the parameters are only saved, so where and how to use it?

1 FFP_OPT_CATEGORY_FORMAT 参数的应用：

It can be seen from the previous section that the format related parameters are stored in ffp->format_opts, and its application is to prepareAsync()prepare resources in , and avformat_open_input()set it for FFmpeg internal use when calling FFmpeg to open the input stream.

static int read_thread(void *arg)
{
    
    
	// ...
	err = avformat_open_input(&ic, is->filename, is->iformat, &ffp->format_opts);
	// ...
}

2 FFP_OPT_CATEGORY_CODEC 参数的应用：

From the previous section, we can see that the codec-related parameters are stored in ffp->codec_opts, and its application is prepareAsync()used when preparing resources, calling FFmpeg to open the decoder, and obtaining stream information.

static int stream_component_open(FFPlayer *ffp, int stream_index)
{
    
    
	// ...
	opts = filter_codec_opts(ffp->codec_opts, avctx->codec_id, ic, ic->streams[stream_index], codec);
	// ...
	if ((ret = avcodec_open2(avctx, codec, &opts)) < 0) {
    
    
        goto fail;
    }
}

static int read_thread(void *arg)
{
    
    
	// ...
	if (ffp->find_stream_info) {
    
    
        AVDictionary **opts = setup_find_stream_info_opts(ic, ffp->codec_opts);
        // ...
        do {
    
    
            // ...
            err = avformat_find_stream_info(ic, opts);
        } while(0);
    }
	// ...
}

3 FFP_OPT_CATEGORY_SWS 参数的应用：

It can be seen from the previous section that sws related parameters are stored in ffp->sws_dict, and this kind of parameters are not really used in ijkplayer, and prepareAsync()the parameters are only printed out when preparing resources.

int ffp_prepare_async_l(FFPlayer *ffp, const char *file_name)
{
    
    
	// ...
	av_log(NULL, AV_LOG_INFO, "===== options =====\n");
    ffp_show_dict(ffp, "player-opts", ffp->player_opts);
    ffp_show_dict(ffp, "format-opts", ffp->format_opts);
    ffp_show_dict(ffp, "codec-opts ", ffp->codec_opts);
    ffp_show_dict(ffp, "sws-opts   ", ffp->sws_dict);
    ffp_show_dict(ffp, "swr-opts   ", ffp->swr_opts);
	// ...
}

4 FFP_OPT_CATEGORY_PLAYER 参数的应用：

From the previous section, we can see that player-related parameters are stored in ffp->player_opts, and its application is prepareAsync()stored in the corresponding member variable of FFplayer when preparing resources:

int ffp_prepare_async_l(FFPlayer *ffp, const char *file_name)
{
    
    
	// ...
	av_log(NULL, AV_LOG_INFO, "===== options =====\n");
    ffp_show_dict(ffp, "player-opts", ffp->player_opts);
    // ...
    av_opt_set_dict(ffp, &ffp->player_opts);
	// ...
}

/**
 * Set all the options from a given dictionary on an object.
 *
 * @param obj a struct whose first element is a pointer to AVClass
 * @param options options to process. This dictionary will be freed and replaced
 *                by a new one containing all options not found in obj.
 *                Of course this new dictionary needs to be freed by caller
 *                with av_dict_free().
 *
 * @return 0 on success, a negative AVERROR if some option was found in obj,
 *         but could not be set.
 *
 * @see av_dict_copy()
 */
int av_opt_set_dict(void *obj, struct AVDictionary **options);

av_opt_set_dict()It is a function defined in FFmpeg libavutil/opt.h, as can be seen from its function description, it sets all the options in the dictionary to the incoming object, that is, FFPlayerother modules optionsare actually saved to their respective objects through this function middle. How are they preserved?

You can see the description of the first parameter of this function, by passing in the first element AVClass of the object structure

@param obj a struct whose first element is a pointer to AVClass

Then let's look at FFPlayerthe structure and ijkmedia/ijkplayer/ff_ffplay_def.hdefine it in:

typedef struct FFPlayer {
    
    
    const AVClass *av_class;
    // ...
} FFPlayer;

Looking at av_classthe usage of , in ijkmedia/ijkplayer/ff_ffplay.cthe documentation:

FFPlayer *ffp_create()
{
    
    
    // ...
    ffp->av_class = &ffp_context_class;
	// ...
}

const AVClass ffp_context_class = {
    
    
    .class_name       = "FFPlayer",
    .item_name        = ffp_context_to_name,
    .option           = ffp_context_options,
    .version          = LIBAVUTIL_VERSION_INT,
    .child_next       = ffp_context_child_next,
    .child_class_next = ffp_context_child_class_next,
};

Here we finally see the use of option:

.option           = ffp_context_options,

Next ffp_context_options, ijkmedia/ijkplayer/ff_ffplay_options.hdefine in , you can see the definition and description of our commonly used parameters ( see the following content for complete parameters ):

static const AVOption ffp_context_options[] = {
    
    
    // original options in ffplay.c
    {
    
     "an",                             "disable audio", OPTION_OFFSET(audio_disable),   OPTION_INT(0, 0, 1) },
    {
    
     "vn",                             "disable video", OPTION_OFFSET(video_disable),   OPTION_INT(0, 0, 1) },
    // ...
    {
    
     "framedrop",                      "drop frames when cpu is too slow", OPTION_OFFSET(framedrop),       OPTION_INT(0, -1, 120) },
    // ...
    {
    
     "start-on-prepared",                  "automatically start playing on prepared", OPTION_OFFSET(start_on_prepared),   OPTION_INT(1, 0, 1) },
    // ...
    {
    
     "enable-accurate-seek",                      "enable accurate seek", OPTION_OFFSET(enable_accurate_seek),       OPTION_INT(0, 0, 1) },
    // ...
    {
    
     "mediacodec",                             "MediaCodec: enable H264 (deprecated by 'mediacodec-avc')", OPTION_OFFSET(mediacodec_avc),          OPTION_INT(0, 0, 1) },
    // ...
    {
    
     "opensles",                           "OpenSL ES: enable", OPTION_OFFSET(opensles),            OPTION_INT(0, 0, 1) },
    {
    
     "soundtouch",                           "SoundTouch: enable", OPTION_OFFSET(soundtouch_enable),OPTION_INT(0, 0, 1) },
    // ...
}

Each of these lines has the following structure:
{name, help, offset, type, default_val, min, max, …}

• name : is the key of the parameter to be set
• help : help description for this parameter
• offset : The offset of the member variable corresponding to the current parameter in the structure, so that the parameter value corresponding to the key in the dictionary can be stored in the member variable corresponding to the structure object when calling av_opt_set_dict(). For example, if the "opensles" option is set outside, then it can OPTION_OFFSET(opensles)be mapped to the opensles member variable in the FFplayer structure according to the offset, and the parameter settings in FFmpeg are all in this mode.

  typedef struct FFPlayer {
        
        
      const AVClass *av_class;
      // ...
      int opensles;
  	int soundtouch_enable;
  	// ...
  } FFPlayer;
  

• type : the data type of the parameter
• default_val : the default value of the parameter
• min : the minimum value of the parameter
• max : the maximum value of the parameter

5 FFP_OPT_CATEGORY_SWR 参数的应用：

It can be seen from the previous section that sws related parameters are stored in ffp->swr_opts, and this type of parameter is not really used in ijkplayer, and this type is not defined in IjkMediaPlayer on the upper layer of Java.

3. The five types of parameter settings of ijkplayer and the functions of each parameter

FFP_OPT_CATEGORY_FORMAT, FFP_OPT_CATEGORY_CODEC, FFP_OPT_CATEGORY_SWS, FFP_OPT_CATEGORY_PLAYER, FFP_OPT_CATEGORY_SWRthese five categories are actually only FFP_OPT_CATEGORY_PLAYERdesigned by ijkplayer with reference to the FFmpeg parameter structure, and the other four are the original parameters of FFmpeg.

1 FFP_OPT_CATEGORY_FORMAT 完整参数：

It is a parameter related to FFmpeg libavformat module, and its complete parameters can be found in the FFmpeg source code: libavformat/options_table.h

https://github.com/bilibili/FFmpeg/blob/ff4.0--ijk0.8.8--20210426--001/libavformat/options_table.h

static const AVOption avformat_options[] = {
    
    
{
    
    "avioflags", NULL, OFFSET(avio_flags), AV_OPT_TYPE_FLAGS, {
    
    .i64 = DEFAULT }, INT_MIN, INT_MAX, D|E, "avioflags"},
{
    
    "direct", "reduce buffering", 0, AV_OPT_TYPE_CONST, {
    
    .i64 = AVIO_FLAG_DIRECT }, INT_MIN, INT_MAX, D|E, "avioflags"},
{
    
    "probesize", "set probing size", OFFSET(probesize), AV_OPT_TYPE_INT64, {
    
    .i64 = 5000000 }, 32, INT64_MAX, D},
{
    
    "formatprobesize", "number of bytes to probe file format", OFFSET(format_probesize), AV_OPT_TYPE_INT, {
    
    .i64 = PROBE_BUF_MAX}, 0, INT_MAX-1, D},
{
    
    "packetsize", "set packet size", OFFSET(packet_size), AV_OPT_TYPE_INT, {
    
    .i64 = DEFAULT }, 0, INT_MAX, E},
// ...
}

2 FFP_OPT_CATEGORY_CODEC 完整参数：

It is the relevant parameter of FFmpeg libavcodec module, its complete parameters can be found in the FFmpeg source code: libavcodec/options_table.h

https://github.com/bilibili/FFmpeg/blob/ff4.0--ijk0.8.8--20210426--001/libavcodec/options_table.h

static const AVOption avcodec_options[] = {
    
    
{
    
    "b", "set bitrate (in bits/s)", OFFSET(bit_rate), AV_OPT_TYPE_INT64, {
    
    .i64 = AV_CODEC_DEFAULT_BITRATE }, 0, INT64_MAX, A|V|E},
{
    
    "ab", "set bitrate (in bits/s)", OFFSET(bit_rate), AV_OPT_TYPE_INT64, {
    
    .i64 = 128*1000 }, 0, INT_MAX, A|E},
// ...
{
    
    "skip_loop_filter", "skip loop filtering process for the selected frames", OFFSET(skip_loop_filter), AV_OPT_TYPE_INT, {
    
    .i64 = AVDISCARD_DEFAULT }, INT_MIN, INT_MAX, V|D, "avdiscard"},
{
    
    "skip_idct"       , "skip IDCT/dequantization for the selected frames",    OFFSET(skip_idct),        AV_OPT_TYPE_INT, {
    
    .i64 = AVDISCARD_DEFAULT }, INT_MIN, INT_MAX, V|D, "avdiscard"},
{
    
    "skip_frame"      , "skip decoding for the selected frames",               OFFSET(skip_frame),       AV_OPT_TYPE_INT, {
    
    .i64 = AVDISCARD_DEFAULT }, INT_MIN, INT_MAX, V|D, "avdiscard"},

// ...
}

3 FFP_OPT_CATEGORY_SWS 完整参数：

It is a parameter related to FFmpeg libswscale module, and its complete parameters can be found in the FFmpeg source code: libswscale/options.c

https://github.com/bilibili/FFmpeg/blob/ff4.0--ijk0.8.8--20210426--001/libswscale/options.c
static const AVOption swscale_options[] = {
    
    
    {
    
     "sws_flags",       "scaler flags",                  OFFSET(flags),     AV_OPT_TYPE_FLAGS,  {
    
     .i64  = SWS_BICUBIC        }, 0,      UINT_MAX,        VE, "sws_flags" },
    {
    
     "fast_bilinear",   "fast bilinear",                 0,                 AV_OPT_TYPE_CONST,  {
    
     .i64  = SWS_FAST_BILINEAR  }, INT_MIN, INT_MAX,        VE, "sws_flags" },
    {
    
     "bilinear",        "bilinear",                      0,                 AV_OPT_TYPE_CONST,  {
    
     .i64  = SWS_BILINEAR       }, INT_MIN, INT_MAX,        VE, "sws_flags" },
    // ...
}

4 FFP_OPT_CATEGORY_PLAYER 完整参数：

It is the relevant parameter of ijkplayer's own configuration, and its complete parameters can be found in the source code of ijkplayer: ijkmedia/ijkplayer/ff_ffplay_options.h

https://github.com/Bilibili/ijkplayer/blob/k0.8.8/ijkmedia/ijkplayer/ff_ffplay_options.h
static const AVOption ffp_context_options[] = {
    
    
    // original options in ffplay.c
    {
    
     "an",                             "disable audio", OPTION_OFFSET(audio_disable),   OPTION_INT(0, 0, 1) },
    {
    
     "vn",                             "disable video", OPTION_OFFSET(video_disable),   OPTION_INT(0, 0, 1) },
    // ...
    {
    
     "framedrop",                      "drop frames when cpu is too slow", OPTION_OFFSET(framedrop),       OPTION_INT(0, -1, 120) },
    // ...
    {
    
     "start-on-prepared",                  "automatically start playing on prepared", OPTION_OFFSET(start_on_prepared),   OPTION_INT(1, 0, 1) },
    // ...
    {
    
     "enable-accurate-seek",                      "enable accurate seek", OPTION_OFFSET(enable_accurate_seek),       OPTION_INT(0, 0, 1) },
    // ...
    {
    
     "mediacodec",                             "MediaCodec: enable H264 (deprecated by 'mediacodec-avc')", OPTION_OFFSET(mediacodec_avc),          OPTION_INT(0, 0, 1) },
    // ...
    {
    
     "opensles",                           "OpenSL ES: enable", OPTION_OFFSET(opensles),            OPTION_INT(0, 0, 1) },
    {
    
     "soundtouch",                           "SoundTouch: enable", OPTION_OFFSET(soundtouch_enable),OPTION_INT(0, 0, 1) },
    // ...
}

5 FFP_OPT_CATEGORY_SWR 完整参数：

It is the relevant parameter of FFmpeg libswresample module, its complete parameters can be found in the FFmpeg source code: libswresample/options.c

https://github.com/bilibili/FFmpeg/blob/ff4.0--ijk0.8.8--20210426--001/libswresample/options.c
static const AVOption options[]={
    
    
{
    
    "ich"                  , "set input channel count"     , OFFSET(user_in_ch_count  ), AV_OPT_TYPE_INT, {
    
    .i64=0                    }, 0      , SWR_CH_MAX, PARAM},
{
    
    "in_channel_count"     , "set input channel count"     , OFFSET(user_in_ch_count  ), AV_OPT_TYPE_INT, {
    
    .i64=0                    }, 0      , SWR_CH_MAX, PARAM},
{
    
    "och"                  , "set output channel count"    , OFFSET(user_out_ch_count ), AV_OPT_TYPE_INT, {
    
    .i64=0                    }, 0      , SWR_CH_MAX, PARAM},
{
    
    "out_channel_count"    , "set output channel count"    , OFFSET(user_out_ch_count ), AV_OPT_TYPE_INT, {
    
    .i64=0                    }, 0      , SWR_CH_MAX, PARAM},
// ...
}