android MediaPlayer NuPlayer parsing and Android MediaPlayerService

MediaPlayer

MediaPlayer generally call logic, the constructor -> setDataSource -> SetVideoSurfaceTexture-> prepare / prepareAsync -> start-> stop-> reset-> destructor, according to the actual demand will invoke pause, isPlaying, getDuration, getCurrentPosition, setLooping , seekTo and other methods.


. 1, the MediaPlayer state in FIG.
L the Idle state
Call new or reset () method to enter idle after creating MediaPlayer
l End state calls release () after the end of l Error state


Playback controls the operation of an error or invalid state call playback control operation
l Initialized state to complete initialization after calling setDataSource l the Prepared state synchronization prepare () or asynchronous prepareAsync () to complete the preparation l the Preparing state is a transient state, it will first call when prepareAsync () this state is entered l Started state to start playing you must call the start () l paused state calls the pause () playback can be paused and after a successful return l stopped state calls stop () stops playing l PlaybackCompleted state when the player reaches the end of the stream, play complete 2, MediaPlayer instance obtain  used directly to create new MediaPlayer MP = new MediaPlayer ();  use create to create, this method no longer call setDataSource



















MP = MediaPlayer.create MediaPlayer (the this, R.raw.test);


3, MediaPlayer and MediaPlayerService
3.1, MediaPlayerService service
after mediaserver will start a number of media-related services add to servicemanager, where there mediaPlayerService. Before this application is started, the system will have a mediaPlayerService this service program. [Main_mediaserver.cpp]

3.2, create MediaPlayer
l Create a MediaPlayer object in the Java application MediaPlayer mediaPlayer = new new MediaPlayer ();
l constructor MediaPlayer's more important is that local native function: native_setup, the corresponding function is
android_media_MediaPlayer_native_setup 

construct Native when the object layer MediaPlayer MediaPlayer.cpp [], the object is also configured parent class. There is a very important way getMediaPlayerService () in the parent class IMediaDeathNotifier MediaPlayer acquired MediaPlayerService, the relationship between the communication MediaPlayer and MediaPlayerService.

3.3, setDataSource set the playback resources

In the process of the entire application, Mediaplayer.cpp in setDataSource get from the service manager in mediaPlayerService service, and then to create a player through the service, this player is a true example of the player, but also the MediaPlayer and MediaPlayerService established contacts.
SetDataSource MediaPlayer.java layer in the final java calls _setDataSource method, a method corresponding to native setDataSource layer of MediaPlayer.cpp.
          

By getMediaPlayerService get BpMediaPlayerService type of service, and mediaPlayerService process BnMediaPlayerService corresponding binder responsible for communication.

                     

In the create function to create an instance of MediaPlayerService :: Client is MediaPlayerService inner class, that is to say MediaPlayerService will be for each client to create a corresponding application process MediaPlayerService :: Client instance , to achieve control of playback and playback , MediaPlayer send a notification to the event. Here, the object in the Server side now created.

    Then MediaPlayer.cpp get a player in an instance sever end, it makes no difference on usage and other instances of local classes, but in fact it is run by a binder mechanism in another process. After obtaining this example continues player-> setDataSource operation.

总结：Java应用程序中使用MediaPlayer.java的setDataSource()会传递到Native层中MediaPlayer.cpp的setDataSource()去执行，而MediaPlayer.cpp又会把这个方法交给MediaPlayerservice去执行。MediaPlayerService则是使用NuPlayer实现的，最后， setDataSource还是交给了NuPlayer去执行了。这个过程把MediaPlayer和MediaPlayerService之间的联系建立起来，同时又把MediaPlayerService和NuPlayer的关系建立了起来。

NuPlayer

NuPlayer由NuPlayerDriver封装，利用了底层的ALooper/AHandler机制来异步地处理请求，ALooper保存消息请求，然后在AHandler中处理。另外，NuPlayer中利用到了Acodec。


1、整体框架图
 
其中的几个部分：
 NuPlayer::Source 解析模块（parser，功能类似FFmpeg的avformat）。其接口与MediaExtractor和
MediaSource组合的接口差不多，同时提供了用于快速定位的seekTo接口。
 NuPlayer::Decoder 解码模块（decoder，功能类似FFmpeg的avcodec），封装了用于AVC、AAC解码的接口，
通过ACodec实现解码（包含OMX硬解码和软解码）。
 NuPlayer::Render 渲染模块（render，功能类似声卡驱动和显卡驱动），主要用于音视频渲染和同步，与
NativeWindow有关。
 NuPlayer 是播放框架中连接Source、Decoder、Renderer的纽带
 NuPlayerDriver 作为NuPlayer类的封装，直接调用NuPlayer。

2、ALooper/AHandler机制

NuPlayer本身继承自AHandler类，而ALooper对象保存在NuPlayerDriver中。ALooper/AHandler机制是一种消息循环处理方式，通常有三个主要部分：消息（message，通常包含Handler）、消息队列（queue）、消息处理线程（looperthread）。此机制会将变量封装到一个消息AMessage结构体中，然后放到队列中去，后台专门有一个线程会从这个队列中取出消息然后通过函数onMessageReceived执行。

 

2.1、AHandler

是消息处理类的父类，基本上只有一个onMessageReceived用于子类继承，deliverMessage用于给类AMessage使用，setID用于给友元类ALooperRoster使用（消息的发送和取出都是调用辅助类ALooperRoster完成的）。

 

2.2、AMessage

    其是消息的载体，在使用AMessage时只需要指定消息类别和要处理该消息的AHandler即可，可以通过构造函数创建，也可以单独调用setWhat和setTarget接口来设置。AMessage构造完成之后，可以调用setXXX设置对应的参数，通过findXXX获取传递的参数。最后通过post即可将消息投递到AHandler的消息队列中。

 

2.3、ALooper及后台线程

是消息处理循环，其调用逻辑是先创建一个ALooper对象，然后调用setName和start接口，之后调用registerHandler设置一个AHandler，这样就完成了初始化。在析构之前需要调用stop接口。

 

2.4、例子

2.5、总结

总的来说整个消息的处理过程就是：在ALoop中的ALooper::start接口会启动一个线程LooperThread，并调用ALooper::loop函数来完成消息的实际执行。而消息是由AMessage通过调用ALooper::post接口，将AMessage添加到消息队列中的。ALooper中存在的后台线程线程LooperThread维护着这个消息队列，线程函数不断从这个队列中取出消息执行。其流程如下图所示：

frameworks/av/media/libmediaplayerservice/MediaPlayerService.h

MediaPlayerService 类的继承关系如下:

Client 和 AudioOutput 是 MediaPlayerService的内部类
IMediaPlayerService是对为提供的接口类, 定义如下:

frameworks/av/include/media/IMediaPlayerService.h
43class IMediaPlayerService: public IInterface
44{
45public:
46    DECLARE_META_INTERFACE(MediaPlayerService);
47
48    virtual sp<IMediaRecorder> createMediaRecorder(const String16 &opPackageName) = 0;
49    virtual sp<IMediaMetadataRetriever> createMetadataRetriever() = 0;
50    virtual sp<IMediaPlayer> create(const sp<IMediaPlayerClient>& client,
51            audio_session_t audioSessionId = AUDIO_SESSION_ALLOCATE) = 0;
52    virtual sp<IOMX>            getOMX() = 0;
53    virtual sp<IHDCP>           makeHDCP(bool createEncryptionModule) = 0;
54    virtual sp<IMediaCodecList> getCodecList() const = 0;
55
56    // Connects to a remote display.
57    // 'iface' specifies the address of the local interface on which to listen for
58    // a connection from the remote display as an ip address and port number
59    // of the form "x.x.x.x:y".  The media server should call back into the provided remote
60    // display client when display connection, disconnection or errors occur.
61    // The assumption is that at most one remote display will be connected to the
62    // provided interface at a time.
63    virtual sp<IRemoteDisplay> listenForRemoteDisplay(const String16 &opPackageName,
64            const sp<IRemoteDisplayClient>& client, const String8& iface) = 0;
65
66    // codecs and audio devices usage tracking for the battery app
67    enum BatteryDataBits {
68        // tracking audio codec
69        kBatteryDataTrackAudio          = 0x1,
70        // tracking video codec
71        kBatteryDataTrackVideo          = 0x2,
72        // codec is started, otherwise codec is paused
73        kBatteryDataCodecStarted        = 0x4,
74        // tracking decoder (for media player),
75        // otherwise tracking encoder (for media recorder)
76        kBatteryDataTrackDecoder        = 0x8,
77        // start to play an audio on an audio device
78        kBatteryDataAudioFlingerStart   = 0x10,
79        // stop/pause the audio playback
80        kBatteryDataAudioFlingerStop    = 0x20,
81        // audio is rounted to speaker
82        kBatteryDataSpeakerOn           = 0x40,
83        // audio is rounted to devices other than speaker
84        kBatteryDataOtherAudioDeviceOn  = 0x80,
85    };
86
87    virtual void addBatteryData(uint32_t params) = 0;
88    virtual status_t pullBatteryData(Parcel* reply) = 0;
89};
  
  

  
  

   
   
1
   
   
2
   
   
3
   
   
4
   
   
5
   
   
6
   
   
7
   
   
8
   
   
9
   
   
10
   
   
11
   
   
12
   
   
13
   
   
14
   
   
15
   
   
16
   
   
17
   
   
18
   
   
19
   
   
20
   
   
21
   
   
22
   
   
23
   
   
24
   
   
25
   
   
26
   
   
27
   
   
28
   
   
29
   
   
30
   
   
31
   
   
32
   
   
33
   
   
34
   
   
35
   
   
36
   
   
37
   
   
38
   
   
39
   
   
40
   
   
41
   
   
42
   
   
43
   
   
44
   
   
45
   
   
46
   
   
47
   
   
48
  
  

MediaPlayerService 是native系统服务, 通过ServiceManager注册到系统中

frameworks/av/media/mediaserver/main_mediaserver.cpp
.....
38    sp<ProcessState> proc(ProcessState::self());
39    sp<IServiceManager> sm(defaultServiceManager());
40    ALOGI("ServiceManager: %p", sm.get());
41    InitializeIcuOrDie();
42    MediaPlayerService::instantiate();
  
  

  
  

   
   
1
   
   
2
   
   
3
   
   
4
   
   
5
   
   
6
   
   
7
  
  

instantiate() 创建MediaPlayerService并注册到系统服务中

frameworks/av/media/libmediaplayerservice/MediaPlayerService.cpp
267void MediaPlayerService::instantiate() {
268    defaultServiceManager()->addService(
269            String16("media.player"), new MediaPlayerService());
270}
  
  

  
  

   
   
1
   
   
2
   
   
3
   
   
4
   
   
5
  
  

MediaPlayerService构造函数如下:

272MediaPlayerService::MediaPlayerService()
273{
274    ALOGV("MediaPlayerService created");
275    mNextConnId = 1;
276
277    mBatteryAudio.refCount = 0;
278    for (int i = 0; i < NUM_AUDIO_DEVICES; i++) {
279        mBatteryAudio.deviceOn[i] = 0;
280        mBatteryAudio.lastTime[i] = 0;
281        mBatteryAudio.totalTime[i] = 0;
282    }
283    // speaker is on by default
284    mBatteryAudio.deviceOn[SPEAKER] = 1;
285
286    // reset battery stats
287    // if the mediaserver has crashed, battery stats could be left
288    // in bad state, reset the state upon service start.
289    BatteryNotifier::getInstance().noteResetVideo();
290    //register built-in player
291    MediaPlayerFactory::registerBuiltinFactories();
292}
  
  

  
  

   
   
1
   
   
2
   
   
3
   
   
4
   
   
5
   
   
6
   
   
7
   
   
8
   
   
9
   
   
10
   
   
11
   
   
12
   
   
13
   
   
14
   
   
15
   
   
16
   
   
17
   
   
18
   
   
19
   
   
20
   
   
21
  
  

registerBuiltinFactories() register the NU_PLAYER and TEST_PLAYER

243void MediaPlayerFactory::registerBuiltinFactories() {
244    Mutex::Autolock lock_(&sLock);
245
246    if (sInitComplete)
247        return;
248
249    registerFactory_l(new NuPlayerFactory(), NU_PLAYER);
250    registerFactory_l(new TestPlayerFactory(), TEST_PLAYER);
251
252    sInitComplete = true;
253}
  
  

  
  

   
   
1
   
   
2
   
   
3
   
   
4
   
   
5
   
   
6
   
   
7
   
   
8
   
   
9
   
   
10
   
   
11
  
  

createMediaRecorder() 创建Recorder用于录音

299sp<IMediaRecorder> MediaPlayerService::createMediaRecorder(const String16 &opPackageName)
300{
301    pid_t pid = IPCThreadState::self()->getCallingPid();
302    sp<MediaRecorderClient> recorder = new MediaRecorderClient(this, pid, opPackageName);
303    wp<MediaRecorderClient> w = recorder;
304    Mutex::Autolock lock(mLock);
305    mMediaRecorderClients.add(w);
306    ALOGV("Create new media recorder client from pid %d", pid);
307    return recorder;
308}
  
  

  
  

   
   
1
   
   
2
   
   
3
   
   
4
   
   
5
   
   
6
   
   
7
   
   
8
   
   
9
   
   
10
  
  

创建Client主要用于播放

325sp<IMediaPlayer> MediaPlayerService::create(const sp<IMediaPlayerClient>& client,
326        audio_session_t audioSessionId)
327{
328    pid_t pid = IPCThreadState::self()->getCallingPid();
329    int32_t connId = android_atomic_inc(&mNextConnId);
330
331    sp<Client> c = new Client(
332            this, pid, connId, client, audioSessionId,
333            IPCThreadState::self()->getCallingUid());
334
335    ALOGV("Create new client(%d) from pid %d, uid %d, ", connId, pid,
336         IPCThreadState::self()->getCallingUid());
337
338    wp<Client> w = c;
339    {
340        Mutex::Autolock lock(mLock);
341        mClients.add(w);
342    }
343    return c;
344}
  
  

  
  

   
   
1
   
   
2
   
   
3
   
   
4
   
   
5
   
   
6
   
   
7
   
   
8
   
   
9
   
   
10
   
   
11
   
   
12
   
   
13
   
   
14
   
   
15
   
   
16
   
   
17
   
   
18
   
   
19
   
   
20
  
  

567MediaPlayerService::Client::Client(
568        const sp<MediaPlayerService>& service, pid_t pid,
569        int32_t connId, const sp<IMediaPlayerClient>& client,
570        audio_session_t audioSessionId, uid_t uid)
571{
572    ALOGV("Client(%d) constructor", connId);
573    mPid = pid;
574    mConnId = connId;
575    mService = service;
576    mClient = client;
577    mLoop = false;
578    mStatus = NO_INIT;
579    mAudioSessionId = audioSessionId;
580    mUID = uid;
581    mRetransmitEndpointValid = false;
582    mAudioAttributes = NULL;
583
584#if CALLBACK_ANTAGONIZER
585    ALOGD("create Antagonizer");
586    mAntagonizer = new Antagonizer(notify, this);
587#endif
588}
  
  

  
  

   
   
1
   
   
2
   
   
3
   
   
4
   
   
5
   
   
6
   
   
7
   
   
8
   
   
9
   
   
10
   
   
11
   
   
12
   
   
13
   
   
14
   
   
15
   
   
16
   
   
17
   
   
18
   
   
19
   
   
20
   
   
21
   
   
22
  
  

      <link href="https://csdnimg.cn/release/phoenix/mdeditor/markdown_views-258a4616f7.css" rel="stylesheet">
              </div>

frameworks/av/media/libmediaplayerservice/MediaPlayerService.h