ijkplayer is a more superior open-source Android / IOS cross-platform player, based ffplay, API for easy integration, customizable compiled volume control.

Based 0.8.8 version ijkplayer, analyze its source, when it comes to handling the package or the interface under different platforms are Android Example.

ijkplayer android integrates three players to achieve:

AndroidMediaPlayer : i.e. Andrews player comes the MediaPlayer , based MediaCodec, AudioTrack other Android API.
IjkExoMediaPlayer : That Google's new ExoPlayer , also based MediaCodec, AudioTrack such as Android API, but compared to the MediaPlayer has the support DASH, senior HLS, custom extensions and so on.
IjkMediaPlayer : Based on the FFmpeg ffplay , MediaCodec integrated hardware decoder, Opengl rendering the like.

In general, ijkplayer refers IjkMediaPlayer , the object of this analysis is IjkMediaPlayer.

Directory Structure

ijkplayer (project folder) 
	├──tools - initialization script project 
	├──config - configuration file used to compile ffmpeg 
	├──extra - where the compiler ijkplayer required dependencies source files, such as ffmpeg, openssl and other 
	├──ijkmedia - the core code 
		├──ijkplayer - player data download and decode related 
		├──ijksdl - rendering audio and video data associated 
	├──android - upper interface packages and related methods on the internet android platform 
	├──ios - the on iOS upper interface packages and internet related methods

Platform to achieve functional differences

IOS and Android platform differences mainly in

Video hardware decoding
Audio rendering
Video Rendering

Platform	Hardware Codec	Video Render	Audio Output
Android	MediaCodec	OpenGL ES, MediaCodec	OpenSL ES、AudioTrack
iOS	VideoToolBox	OpenGL ES	AudioQueue

IjkMediaPlayer and Native interaction

Playback control related to start、pause、stopsuch a method call the corresponding native

The bottom of the status reporting information (such as playing status underlying callback) related postEventFromNative, these methods take the initiative to call the bottom (with @CalledByNativecomment)

initialization

public final class  extends AbstractMediaPlayer {
    private void initPlayer(IjkLibLoader libLoader) {
        loadLibrariesOnce(libLoader);
        initNativeOnce();

        Looper looper;
        if ((looper = Looper.myLooper()) != null) {
            mEventHandler = new EventHandler(this, looper);
        } else if ((looper = Looper.getMainLooper()) != null) {
            mEventHandler = new EventHandler(this, looper);
        } else {
            mEventHandler = null;
        }

        
         * Native setup requires a weak reference to our object. It's easier to
         * create it here than in C++.
         */
        native_setup(new WeakReference<IjkMediaPlayer>(this));
    }
}

initPlayer 方法，共做了四件事：

加载 so 库
静态初始化底层，底层其实什么都没做
初始化 Message Handler，处理底层状态信息的上报
初始化底层，这部分做的工作最多

初始化底层

c代码

ijkplayer/android/ijkplayer/ijkplayer-armv7a/src/main/jni/ijkmedia/ijkplayer/android/

主要逻辑位于

ijkpalyer_android.c 的 ijkmp_android_create 方法

// 创建底层播放器对象，设置消息处理函数
IjkMediaPlayer *mp = ijkmp_create(msg_loop, saveMode, hard_mux);

// 创建图像渲染对象
mp->ffplayer->vout = SDL_VoutAndroid_CreateForAndroidSurface();

// 初始化视频解码器（软/硬）、音频输出设备（opensles/audioTrack）
mp->ffplayer->pipeline = ffpipeline_create_from_android(mp->ffplayer);

软硬解码选择

跟踪 pipeline/ffpipeline_android.c 的 ffpipeline_create_from_android 方法

发现是用函数指针记录类 IjkMediaPlayer.setOption() 设置的属性

配置

初始化后 IjkMediaPlayer 后，可以对其进行一系列配置，例如:

// 设置硬解码
mIjkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec", 1);

// 设置 opensles 方式输出音频
mIjkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "opensles", 1);

setOption() 会调用到底层 ff_ffplay.c 的 ffp_set_option() 方法

播放

播放器必然是通过多线程同时进行解封装、解码、视频渲染等工作的，对于 Ijkplayer 来说，开辟的线程如下：

当对播放器设置视频源路径、解码方式、输出模式等播放选项后，就可以开始播放了，播放入口方法为 ffp_prepare_async_l，此方法中调用了比较重要的两个方法：

// 打开音频输出设备
   ffp->aout = ffpipeline_open_audio_output(ffp->pipeline, ffp);
   ...
   // 创建音/视频数据解码前/后队列， 创建解封装和视频渲染线程
   VideoState *is = stream_open(ffp, file_name, NULL);

stream_open 方法则相当重要了，梳理一下该方法中涉及到的关键方法：

音视频同步

对于播放器来说，音视频同步是一个关键点，同时也是一个难点，同步效果的好坏，直接决定着播放器的质量。

通常音视频同步的解决方案就是选择一个参考时钟，播放时读取音视频帧上的时间戳，同时参考当前时钟参考时钟上的时间来安排播放。如下图所示：

如果音视频帧的播放时间大于当前参考时钟上的时间，则不急于播放该帧，直到参考时钟达到该帧的时间戳；如果音视频帧的时间戳小于当前参考时钟上的时间，则需要“尽快”播放该帧或丢弃，以便播放进度追上参考时钟。

参考时钟的选择

有多种方式：

选取视频时间戳作为参考时钟源
选取音频时间戳作为参考时钟源
选取外部时间作为参考时钟源

考虑人对视频、和音频的敏感度，在存在音频的情况下，优先选择音频作为主时钟源。

ijkplayer the default is the case of using audio as a reference source

Event Processing

During playback, complete or change certain behaviors, such as prepare complete, start rendering, it is necessary to inform the external form of an event, so that the upper layer to make specific business processes.

Underlying message reporting event when the player is actually to be transmitted into the message queue, and another thread will continue to get the message from the queue is notified to the outside

Reference & Extended

Original: Big Box ijkplayer analysis

ijkplayer analysis