In the Android multimedia framework, stagefrightplayer is a package of Awesomeplayer is AwesomePlayer agent, so there is real work, of course, today's hero -AwesomePlayer. AwesomePlayer Plainly also an ordinary player, he and open-source structure VCL, mplayer, ffmpeg, etc. is the same, only be achieved in different ways, here the following four steps to achieve AwesomePlayer of analysis.
First, the basic model multimedia player
Two, AwesomePlayer overall structure
Three, AwesomePlayer drive way
Fourth, the analysis of key members AwesomePlayer
Five, AwesomePlayer code analysis of the implementation process
First, the basic model multimedia player
1. source: data sources, data source is not necessarily a local file, there may be a variety of protocols on the network such as: http, rtsp, HLS and so on. source task is to abstract the data source provides steady stream of data that it needs for the next demux module. demux do not care data in the end is to come from somewhere.
2. demux demultiplexing: video file under normal circumstances are the audio and video ES stream by certain rules interwoven together. This rule is the rule container. There are many different container formats. Such as ts, mp4, flv, mkv, avi, rmvb and so on. ES demux function is the audio-video stream from the stripping vessel, and then are sent to different decoders. In fact, the audio and the video itself is two separate systems. Container package them together. But they are decoded independently, so before decoding, respectively, it needs to be independent. demux is to do this live, he provided a decoder to decode the data stream for the next step.
3. decoder decoder: decoder - player of the core module. Divided into audio and video decoder. Image after recording, the original audio and video are up a lot of space, but also a high degree of redundancy of data. Therefore, it will usually be some kind of compression (compression technology is the data redundancy in the production of remove the correlation between data). this is our well-known audio and video encoding formats, including MPEG1 (VCD) / MPEG2 (DVD ) / MPEG4 / H.264 , etc. the role of audio and video decoder is these compressed data restore the original audio and video data. Of course, the process of encoding and decoding are substantially lossy role of the decoder is to restore the encoded data into the original data.
4. output Output: output section is divided into audio and video output. Audio (pcm) and the decoded video (yuv) of raw data need to be supported in order to output module audio and video real people's sensory system (eyes and ears) to identify.
Therefore, a multimedia player described above is roughly divided into four parts. How to achieve the four most abstract, and to find a reasonable way to organize these sections and in motion. It is different implementations of each player only.
Two, AwesomePlayer overall structure
mExtractor belongs MediaExtractor category for each media file format, are required to achieve a subclass MediaExtractor, e.g. AVI extractor class file is AVIExtractor, complete parsing the data in the constructor, the primary information: the flowing media files number, the sampling rate of the audio stream, the channel number, quantization bit number, etc., of the video stream width, height, frame rate and other information.
mAudioTrack and mVideoTrack belonging MediaSource classes, and mAudioTrack mVideoTrack onPrepareAsyncEvent when an event is triggered by the MediaExtractor separated out. For each stream corresponds to a single MediaSource, avi to an example, as AVISource, MediaSource provides an interface state (start stop ), the data read interface (read), the parameter query interface (getFormat). Wherein the read function call time, can be considered a data packet corresponding to the read. In general, since MediaExtractor responsible for parsing, so MediaSource are generally obtained by the read operation of the interface MediaExtractor offset and size, MediaSource performed only for the data read operation, does not participate in parsing, and of VideoTrack AudioTrack refers Extractor (i.e. demux ) of the two channels, respectively, from the output here is the simple demultiplexing Video / Audio streaming.
mAudioSource 和mVideoSource也属于MediaSource类，这里mAudioSource &mVideoSource可以认为是awesomeplayer与decoder之间的桥梁，awesomeplayer从mAudioSource &mVideoSource 里获取数据，进行播放，而decoder则负责解码并向mAudioSource &mVideoSource 填充数据， 这里awesomeplayer与decoder的通信是通过OMXClient mClient; 成员进行的，OMX*解码器是一个服务，每个awesomeplayer对象都包含一个单独的client端与之交互。
mAudioPlayer属于AudioPlayer类，mVideoRenderer属于AwesomeRenderer类，mAudioPlayer 是负责音频输出的模块，主要封装关系为：mAudioPlayer->mAudioSink->mAudioTrack(这里的mAudioTrack 与前面不同，此处为AudioTrack对象)实际进行音频播放的对象为mAudioTrack-audioflinger 结构，mVideoRenderer 是负责视频显示的模块，封装关系为：mVideoRenderer ->mTarget（SoftwareRenderer）->mNativeWindow ，即VideoRenderer + Surface即视频的输出，AudioSink即音频的输出.
三、AwesomePlayer的驱动方式
这里就不得不提到android中的一个类即TimedEventQueue，这是一个消息处理类， 这里通过对每个事件消息提供一个fire函数完成相应的操作。而整个播放过程的驱动方式为递归的触发mVideoEvent 事件来控制媒体文件的播放。TimedEventQueue资料建议自行查找下。
说明：详细可参考下面的代码分析
awesomeplayer中主要有如下几个事件
sp<TimedEventQueue::Event> mVideoEvent = new AwesomeEvent(this, &AwesomePlayer::onVideoEvent); --- 显示一帧画面，并负责同步处理
sp<TimedEventQueue::Event> mStreamDoneEvent = new AwesomeEvent(this, &AwesomePlayer::onStreamDone); -- 播放结束的处理
sp<TimedEventQueue::Event> mBufferingEvent = new AwesomeEvent(this, &AwesomePlayer::onBufferingUpdate); -- cache数据
sp<TimedEventQueue::Event> mCheckAudioStatusEvent = new AwesomeEvent(this, &AwesomePlayer::onCheckAudioStatus); -- 监测audio 状态：seek以及播放结束
sp<TimedEventQueue::Event> mVideoLagEvent = new AwesomeEvent(this, &AwesomePlayer::onVideoLagUpdate); -- 监测video 解码性能
sp<TimedEventQueue::Event> mAsyncPrepareEvent = new AwesomeEvent(this, &AwesomePlayer::onPrepareAsyncEvent); -- 完成prepareAsync工作
四、AwesomePlayer的关键成员分析
关键成员之demux相关
sp<MediaSource> mVideoTrack;
sp<MediaSource> mAudioTrack分别代表一个视频轨道和音频轨道,用于提取视频流和音频流(Demux后但未解码的数据). mVideoTrack和mAudioTrack在 onPrepareAsyncEvent事件被触发时,由MediaExtractor分离出来.
void AwesomePlayer::onPrepareAsyncEvent() {
status_t err = finishSetDataSource_l();
}
status_t AwesomePlayer::finishSetDataSource_l() {
sp<MediaExtractor> extractor = MediaExtractor::Create(dataSource);
return setDataSource_l(extractor);
}
status_t AwesomePlayer::setDataSource_l(const sp<MediaExtractor> &extractor) {
for (size_t i = 0; i < extractor->countTracks(); ++i) {
sp<MetaData> meta = extractor->getTrackMetaData(i);
const char *mime;
CHECK(meta->findCString(kKeyMIMEType, &mime));
if (!haveVideo && !strncasecmp(mime, "video/", 6)) {
setVideoSource(extractor->getTrack(i)); //
haveVideo = true;
} else if (!haveAudio && !strncasecmp(mime, "audio/", 6)) {
setAudioSource(extractor->getTrack(i));
haveAudio = true;
if (haveAudio && haveVideo) {
break;
}
}
} 这里依据提供的dataSource建立对应的MediaExtractor，这里说明下，有了dataSource，就可以从文件中读取数据，就可以通过分析文件头解析出文件具体是哪种格式，然后建立相应的MediaExtractor，之前有介绍，在MediaExtractor 建立的时候便完成了对文件的解析，包括流数量，各个流的具体信息等，下面就可以直接使用了，得到extractor之后，通过setVideoSource() setAudioSource()产生独立的mVideoTrack(视频)、mAudioTrack(音频)数据流，分别为音视频解码器提供有各自需要的数据流。从上面这段代码可以看到AwesomePlayer默认采用第一个VideoTrack和第一个AudioTrack,
流数量可以通过extractor->countTracks() 得到，每个流对应的信息存储在一个MetaData中通过extractor->getTrackMetaData(i);
获取kKeyMIMEType参数，此参数标示了流类型是音频 视频 还是字幕
关键成员之音频相关
sp<MediaSource> mAudioSource;mAudioSource可以认为是一个音频解码器的封装，
sp<MediaPlayerBase::AudioSink> mAudioSink;
mAudioSink代表一个音频输出设备.用于播放解码后的音频数据
AudioPlayer *mAudioPlayer;mAudioPlayer把mAudioSource和mAudioSink包起来,完成一个音频播放器的功能.如start, stop, pause, seek 等.
AudioPlayer和AudioSink通过Callback建立关联.当AudioSink可以输出音频时,会通过回调通知AudioPlayer填充音频数据.而此时AudioPlayer会尝试从AudioSource读取音频数据.
关键成员之视频相关
sp<MediaSource> mVideoSourcemVideoSource可以认为是一个视频解码器的封装,用于产生视频图像供AwesomeRender渲染, mVideoSource的数据源则由mVideoTrack提供.mVideoSource由OMXCodec创建.
status_t AwesomePlayer::initVideoDecoder(uint32_t flags) {
mVideoSource = OMXCodec::Create(
mClient.interface(), mVideoTrack->getFormat(),
false, // createEncoder
mVideoTrack,
NULL, flags);
} sp<AwesomeRenderer> mVideoRenderer mVideoRenderer负责将解码后的图片渲染输出
sp<ISurface> mISurfacemISurface供播放器渲染的画布
关键数据之其他
OMXClient mClientOMX可以理解为一个编解码器的库,AwesomePlayer利用OMXClient跟OMXIL进行通信.这里OMX IL类似于一个服务端. AwesomePlayer作为一个客户端,请求OMX IL进行解码的工作.可参考下图：
TimedEventQueue mQueueTimedEventQueue一个消息队列类，可以认为TimeEventQueue 是一个调度系统，调度的对象是事件Event，一个TimeEventQueue 可以同时处理多个事件TimedEventQueue中维护了一个队列，外面通过调用其提供的方法postEvent, postEventWithDelay等等来向队列添加事件，在执行完mQueue.start()后，TimedEventQueue将启动一个线程，用来取出队列中的事件，并执行之
AwesomePlayer采用定时器队列的方式进行运作. mQueue在MediaPlayer调用 prepare的时候就开始运作, (而不是MediaPlayer.start).
status_t AwesomePlayer::prepareAsync() {
if (!mQueueStarted) {
mQueue.start();
mQueueStarted = true;
}
return OK;
} AwesomePlayer处理了几个定时器事件,包括:
(1)onVideoEvent();//
(2)onStreamDone();//
(3)onBufferingUpdate();//
(4)onCheckAudioStatus();//
(5)onPrepareAsyncEvent();//
总结:从关键的成员可以看出,AwesomePlayer拥有视频源和音频源 (VideoTrack, AudioTrack),有音视频解码器(VideoSoure, AudioSource),可以渲染图像(AwesomeRenderer) , 可以输出声音 (AudioSink),具备一个播放器完整的材料了.
五、AwesomePlayer的实现过程代码分析
5.1总的基本播放流程分析
还是从之前的一个播放器的播放序列看起
StagefrightPlayer player =newStagefrightPlayer();
player->setDataSource(*)
player->prepareAsync()
player->start(); ①由于StagefrightPlayer是AwesomePlayer的代理，所以我们接下来看下AwesomePlayer的构造函数
AwesomePlayer::AwesomePlayer()
: mQueueStarted(false),
mUIDValid(false),
mTimeSource(NULL),
mVideoRenderingStarted(false),
mVideoRendererIsPreview(false),
mAudioPlayer(NULL),
mDisplayWidth(0),
mDisplayHeight(0),
mVideoScalingMode(NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW),
mFlags(0),
mExtractorFlags(0),
mVideoBuffer(NULL),
mDecryptHandle(NULL),
mLastVideoTimeUs(-1),
mTextDriver(NULL) {
CHECK_EQ(mClient.connect(), (status_t)OK);
DataSource::RegisterDefaultSniffers();
mVideoEvent =newAwesomeEvent(this, &AwesomePlayer::onVideoEvent);
mVideoEventPending =false;
mStreamDoneEvent =newAwesomeEvent(this, &AwesomePlayer::onStreamDone);
mStreamDoneEventPending =false;
mBufferingEvent =newAwesomeEvent(this, &AwesomePlayer::onBufferingUpdate);
mBufferingEventPending =false;
mVideoLagEvent =newAwesomeEvent(this, &AwesomePlayer::onVideoLagUpdate);
mVideoEventPending =false;
mCheckAudioStatusEvent =newAwesomeEvent(
this, &AwesomePlayer::onCheckAudioStatus);
mAudioStatusEventPending =false;
reset();
}
从上述代码中可见：
在awesomeplayer的构造函数中主要就是做些准备工作：如创立event对象，并提供fire函数，设置对应的各个状态变量
还有一点需要注意的是 mClient.connect() 建立了awesomeplayer与omx解码器之间的链接
②设置数据源URI
status_t AwesomePlayer::setDataSource(
const char *uri, const KeyedVector<String8, String8> *headers) {
mUri = uri; // 这里只是把URL保存起来而已, 真正的工作在Prepare之后进行
return OK;
} ③开启定时器队列,并且 Post一个AsyncPrepareEvent事件
status_t AwesomePlayer::prepareAsync() {
mQueue.start(); // 开启定时器队列
mAsyncPrepareEvent = new AwesomeEvent(
this, &AwesomePlayer::onPrepareAsyncEvent); // Post AsyncPrepare 事件
mQueue.postEvent(mAsyncPrepareEvent);
return OK;
} Prepare之后,AwesomePlayer开始运作.
④AsyncPrepare事件被触发
当这个事件被触发时, AwesomePlayer开始创建 VideoTrack和AudioTrack ,然后创建 VideoDecoder和AudioDecoder
void AwesomePlayer::onPrepareAsyncEvent() {
finishSetDataSource_l(); // a. 创建视频源和音频源
initVideoDecoder(); // b. 创建视频解码器
initAudioDecoder(); // c. 创建音频解码器
} 至此,播放器准备工作完成,可以开始播放了
⑤Post第一个VideoEvent（显示一帧画面并进行同步处理）
调用顺序：AwesomePlayer::play()调用 -> AwesomePlayer::play_l()调用 ->AwesomePlayer::postVideoEvent_l(int64_t delayUs)
void AwesomePlayer::postVideoEvent_l(int64_t delayUs) {
mQueue.postEventWithDelay(mVideoEvent, delayUs < 0 ? 10000 : delayUs);
} ⑥VideoEvent被触发
void AwesomePlayer::onVideoEvent() {
mVideoSource->read(&mVideoBuffer, &options); // 从视频解码器中读出视频图像
if (mVideoRendererIsPreview || mVideoRenderer == NULL) { // 创建AwesomeRenderer (如果没有的话)
initRenderer_l();
}
mVideoRenderer->render(mVideoBuffer); // 渲染视频图像
postVideoEvent(); // 再次发送一个VideoEvent, 这样播放器就不停的播放了
} 总结: SetDataSource -> Prepare -> Play-> postVieoEvent -> OnVideoEvent -> postVideoEvent-> ....onVideoEvent-> postStreamDoneEvent ->播放结束
5.2、视频 /音频的分离
①就像前面基本播放流程分析描述的那样，当AsyncPrepare事件被触发时, AwesomePlayer会调用finishSetDataSource_l创建 VideoTrack 和 AudioTrack，finishSetDataSource_l通过URI前缀判断 媒体类型,比如http, rtsp,或者本地文件等 这里的uri就是一开始 通过setDataSource设置的 根据uri创建相应的DataSource,再进一步的利用 DataSource创建MediaExtractor做A/V分离
status_t AwesomePlayer::finishSetDataSource_l() {
sp<datasource> dataSource;
/// 通过URI前缀判断媒体类型, 比如 http, rtsp,或者本地文件等
/// 这里的uri就是一开始 通过setDataSource设置的
/// 根据uri 创建相应的MediaExtractor
if (!strncasecmp("http://", mUri.string(), 7)) {
mConnectingDataSource = new NuHTTPDataSource;
mConnectingDataSource->connect(mUri, &mUriHeaders);
mCachedSource = new NuCachedSource2(mConnectingDataSource);
dataSource = mCachedSource;
} else if (!strncasecmp("rtsp://", mUri.string(), 7)) {
mRTSPController->connect(mUri.string());
sp<mediaextractor> extractor = mRTSPController.get();
/// rtsp 比较特殊, MediaExtractor对象的创建不需要DataSource
return setDataSource_l(extractor);
} else {
/// 本地文件
dataSource = DataSource::CreateFromURI(mUri.string(), &mUriHeaders);
}
/// 用dataSource创建一个MediaExtractor用于A/V分离
sp<mediaextractor> extractor = MediaExtractor::Create(dataSource);
return setDataSource_l(extractor);
}
</mediaextractor></mediaextractor></datasource>②根据根据DataSource创建MediaExtractor
先来看AwesomePlayer的构造函数,里面有一行代码.
AwesomePlayer::AwesomePlayer(){
//...
DataSource::RegisterDefaultSniffers();
//...
}RegisterDefaultSniffers注册了一些了媒体的MIME类型的探测函数.
void DataSource::RegisterDefaultSniffers() {
RegisterSniffer(SniffMPEG4);
RegisterSniffer(SniffMatroska);
RegisterSniffer(SniffOgg);
RegisterSniffer(SniffWAV);
RegisterSniffer(SniffAMR);
RegisterSniffer(SniffMPEG2TS);
RegisterSniffer(SniffMP3);
}
这些探测用于判断媒体的MIME类型,进而决定要创建什么样的MediaExtractor.
再回到 MediaExtractor::Create, MediaExtractor对象在这里创建.下面代码有点长,其实这段代码只是根据MIME类型创建Extractor的各个分支而已.
sp<mediaextractor> MediaExtractor::Create(
const sp<datasource> &source, const char *mime) {
sp<amessage> meta;
String8 tmp;
if (mime == NULL) {
float confidence;
if (!source->sniff(&tmp, &confidence, &meta)) {
LOGV("FAILED to autodetect media content.");
return NULL;
}
mime = tmp.string();
LOGV("Autodetected media content as '%s' with confidence %.2f",
mime, confidence);
}
bool isDrm = false;
// DRM MIME type syntax is "drm+type+original" where
// type is "es_based" or "container_based" and
// original is the content's cleartext MIME type
if (!strncmp(mime, "drm+", 4)) {
const char *originalMime = strchr(mime+4, '+');
if (originalMime == NULL) {
// second + not found
return NULL;
}
++originalMime;
if (!strncmp(mime, "drm+es_based+", 13)) {
// DRMExtractor sets container metadata kKeyIsDRM to 1
return new DRMExtractor(source, originalMime);
} else if (!strncmp(mime, "drm+container_based+", 20)) {
mime = originalMime;
isDrm = true;
} else {
return NULL;
}
}
MediaExtractor *ret = NULL;
if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_MPEG4)
|| !strcasecmp(mime, "audio/mp4")) {
ret = new MPEG4Extractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_MPEG)) {
ret = new MP3Extractor(source, meta);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_AMR_NB)
|| !strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_AMR_WB)) {
ret = new AMRExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_FLAC)) {
ret = new FLACExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_WAV)) {
ret = new WAVExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_OGG)) {
ret = new OggExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_MATROSKA)) {
ret = new MatroskaExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_MPEG2TS)) {
ret = new MPEG2TSExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_AVI)) {
ret = new AVIExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_WVM)) {
ret = new WVMExtractor(source);
} else if (!strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_AAC_ADTS)) {
ret = new AACExtractor(source);
}
if (ret != NULL) {
if (isDrm) {
ret->setDrmFlag(true);
} else {
ret->setDrmFlag(false);
}
}
return ret;
}
</amessage></datasource></mediaextractor>③根据根据MediaExtractor做A/V分离
再来看看setDataSource_l(const sp&extractor) ,这是A/V分离的最后步骤
status_t AwesomePlayer::setDataSource_l(const sp<mediaextractor> &extractor) {
/// 从全部的Track中选取一个Video Track和一个AudioTrack
for (size_t i = 0; i < extractor->countTracks(); ++i) {
sp<metadata> meta = extractor->getTrackMetaData(i);
const char *mime;
CHECK(meta->findCString(kKeyMIMEType, &mime));
if (!haveVideo && !strncasecmp(mime, "video/", 6)) {
setVideoSource(extractor->getTrack(i));
haveVideo = true;
} else if (!haveAudio && !strncasecmp(mime, "audio/", 6)) {
setAudioSource(extractor->getTrack(i));
haveAudio = true;
}
if (haveAudio && haveVideo) {
break;
}
} /// Flags 标志这个媒体的一些属性:
/// CAN_SEEK_BACKWARD 是否能后退10秒
/// CAN_SEEK_FORWARD 是否能前进10秒
/// CAN_SEEK 能否Seek?
/// CAN_PAUSE 能否暂停
mExtractorFlags = extractor->flags();
return OK;
}从这个函数可以看到MediaExtractor需要实现的基本比较重要的接口 (这个几个接口都是纯虚函数,可见Extractor的子类是一定要搞定它们的)
virtual size_t countTracks() = 0; 该媒体包含了几个Track?
virtual sp getTrack(size_t index) = 0;获取指定的Video/Audio Track,可以看到一个Track本质上就是一个MediaSource
virtual sp getTrackMetaData ( size_t index,uint32_t flags = 0) = 0;获取指定的Track的MetaData.在AwesomePlayer 中, MetaData实际上就是一块可以任意信息字段的叉烧, 字段类型可以是字符串或者是整形等等.这里Track的MetaData包含了Track的MIME类型.这样AwesomePlayer就可以知道这个Track是Video还是Audio的了
总结:至此,AwesomePlayer 就拥有VideoTrack和AudioTrack了 (可能只有VideoTrack或者只有AudioTrack,例如MP3).接下来 音视频解码器VideoSource/AudioSource 将从Video/Audio Track 中读取数据进行解码.
④创建视频解码器
VideoTrack/AudioTrack创建完毕之后, 紧接着就是创建 VideoSource了 (见 1.2.3).看看initVideoDecoder
status_t AwesomePlayer::initVideoDecoder(uint32_t flags) {
mVideoSource = OMXCodec::Create(
mClient.interface(), mVideoTrack->getFormat(),
false, // createEncoder
mVideoTrack,
NULL, flags);
/// ...
return mVideoSource != NULL ? OK : UNKNOWN_ERROR;
}VideoSource是由OMXCodec::Create 创建的.从OMXCodec::Create的参数可以看出创建一个视频解码器需要什么材料:
a. OMXClient.用于跟OMXIL 通讯.假如最后用的是OMXCodec 也不是SoftCodec的话,需要用到它.
b. mVideoTrack->getFormat (). getFormat返回包含该video track格式信息的MetaData.
c. mVideoTrack，解码器会从 Video Track 中读取数据进行解码.
⑤创建视频解码器的函数OMXCodec::Create
sp<mediasource> OMXCodec::Create(
const sp<iomx> &omx,
const sp<metadata> &meta, bool createEncoder,
const sp<mediasource> &source,
const char *matchComponentName,
uint32_t flags) {
/// 获取MIME类型
const char *mime;
bool success = meta->findCString(kKeyMIMEType, &mime);
/// 根据MIME找出可能匹配的Codec
Vector<string8> matchingCodecs;
findMatchingCodecs(
mime, createEncoder, matchComponentName, flags, &matchingCodecs);
IOMX::node_id node = 0;
/// 对每一种可能匹配的Codec, 尝试申请Codec
const char *componentName;
for (size_t i = 0; i < matchingCodecs.size(); ++i) {
componentName = matchingCodecs[i].string();
/// 尝试申请软Codec
sp<mediasource> softwareCodec = createEncoder?
InstantiateSoftwareEncoder(componentName, source, meta):
InstantiateSoftwareCodec(componentName, source);
if (softwareCodec != NULL) {
return softwareCodec;
}
/// 尝试申请OMXCodec
status_t err = omx->allocateNode(componentName, observer, &node);
if (err == OK) {
sp<omxcodec> codec = new OMXCodec(
omx, node, quirks,
createEncoder, mime, componentName,
source);
/// 配置申请出来的OMXCodec
err = codec->configureCodec(meta, flags);
if (err == OK) {
return codec;
}
}
}
return NULL;
}
</omxcodec></mediasource></string8></mediasource></metadata></iomx></mediasource>Create中OMXCodec::findMatchingCodecs函数找出可能匹配的Codec
findMatchingCodecs根据传入的MIME 从kDecoderInfo中找出MIME对于的Codec名 (一种MIME可能对应多种Codec)
void OMXCodec::findMatchingCodecs(
const char *mime,
bool createEncoder, const char *matchComponentName,
uint32_t flags,
Vector<string8> *matchingCodecs) {
for (int index = 0;; ++index) {
const char *componentName;
componentName = GetCodec(
kDecoderInfo,
sizeof(kDecoderInfo) / sizeof(kDecoderInfo[0]),
mime, index);
matchingCodecs->push(String8(componentName));
}
}
</string8>看看 kDecoderInfo里面包含了什么Codec吧,有点长.
static const CodecInfo kDecoderInfo[] = {
{ MEDIA_MIMETYPE_IMAGE_JPEG, "OMX.TI.JPEG.decode" },
// { MEDIA_MIMETYPE_AUDIO_MPEG, "OMX.TI.MP3.decode" },
{ MEDIA_MIMETYPE_AUDIO_MPEG, "MP3Decoder" },
// { MEDIA_MIMETYPE_AUDIO_MPEG, "OMX.PV.mp3dec" },
// { MEDIA_MIMETYPE_AUDIO_AMR_NB, "OMX.TI.AMR.decode" },
{ MEDIA_MIMETYPE_AUDIO_AMR_NB, "AMRNBDecoder" },
// { MEDIA_MIMETYPE_AUDIO_AMR_NB, "OMX.PV.amrdec" },
{ MEDIA_MIMETYPE_AUDIO_AMR_WB, "OMX.TI.WBAMR.decode" },
{ MEDIA_MIMETYPE_AUDIO_AMR_WB, "AMRWBDecoder" },
// { MEDIA_MIMETYPE_AUDIO_AMR_WB, "OMX.PV.amrdec" },
{ MEDIA_MIMETYPE_AUDIO_AAC, "OMX.TI.AAC.decode" },
{ MEDIA_MIMETYPE_AUDIO_AAC, "AACDecoder" },
// { MEDIA_MIMETYPE_AUDIO_AAC, "OMX.PV.aacdec" },
{ MEDIA_MIMETYPE_AUDIO_G711_ALAW, "G711Decoder" },
{ MEDIA_MIMETYPE_AUDIO_G711_MLAW, "G711Decoder" },
{ MEDIA_MIMETYPE_VIDEO_MPEG4, "OMX.qcom.7x30.video.decoder.mpeg4" },
{ MEDIA_MIMETYPE_VIDEO_MPEG4, "OMX.qcom.video.decoder.mpeg4" },
{ MEDIA_MIMETYPE_VIDEO_MPEG4, "OMX.TI.Video.Decoder" },
{ MEDIA_MIMETYPE_VIDEO_MPEG4, "OMX.SEC.MPEG4.Decoder" },
{ MEDIA_MIMETYPE_VIDEO_MPEG4, "OMX.TCC.mpeg4dec" },
{ MEDIA_MIMETYPE_VIDEO_MPEG4, "M4vH263Decoder" },
// { MEDIA_MIMETYPE_VIDEO_MPEG4, "OMX.PV.mpeg4dec" },
{ MEDIA_MIMETYPE_VIDEO_H263, "OMX.qcom.7x30.video.decoder.h263" },
{ MEDIA_MIMETYPE_VIDEO_H263, "OMX.qcom.video.decoder.h263" },
{ MEDIA_MIMETYPE_VIDEO_H263, "OMX.SEC.H263.Decoder" },
{ MEDIA_MIMETYPE_VIDEO_H263, "OMX.TCC.h263dec" },
{ MEDIA_MIMETYPE_VIDEO_H263, "M4vH263Decoder" },
// { MEDIA_MIMETYPE_VIDEO_H263, "OMX.PV.h263dec" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "OMX.qcom.7x30.video.decoder.avc" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "OMX.qcom.video.decoder.avc" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "OMX.TI.Video.Decoder" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "OMX.SEC.AVC.Decoder" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "OMX.TCC.avcdec" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "AVCDecoder" },
// { MEDIA_MIMETYPE_VIDEO_AVC, "OMX.PV.avcdec" },
{ MEDIA_MIMETYPE_AUDIO_VORBIS, "VorbisDecoder" },
{ MEDIA_MIMETYPE_VIDEO_VPX, "VPXDecoder" },
// TELECHIPS, SSG
{ MEDIA_MIMETYPE_AUDIO_MPEG_TCC, "OMX.TCC.mp3dec" },
{ MEDIA_MIMETYPE_AUDIO_AAC_TCC, "OMX.TCC.aacdec" },
{ MEDIA_MIMETYPE_AUDIO_VORBIS_TCC, "OMX.TCC.vorbisdec" },
{ MEDIA_MIMETYPE_AUDIO_WMA, "OMX.TCC.wmadec" },
{ MEDIA_MIMETYPE_AUDIO_AC3, "OMX.TCC.ac3dec" },
{ MEDIA_MIMETYPE_AUDIO_RA, "OMX.TCC.radec" },
{ MEDIA_MIMETYPE_AUDIO_FLAC, "OMX.TCC.flacdec" },
{ MEDIA_MIMETYPE_AUDIO_APE, "OMX.TCC.apedec" },
{ MEDIA_MIMETYPE_AUDIO_MP2, "OMX.TCC.mp2dec" },
{ MEDIA_MIMETYPE_AUDIO_PCM, "OMX.TCC.pcmdec" },
{ MEDIA_MIMETYPE_AUDIO_DTS, "OMX.TCC.dtsdec" },
{ MEDIA_MIMETYPE_VIDEO_VC1, "OMX.TCC.wmvdec" },
{ MEDIA_MIMETYPE_VIDEO_WMV12, "OMX.TCC.wmv12dec" },
{ MEDIA_MIMETYPE_VIDEO_RV, "OMX.TCC.rvdec" },
{ MEDIA_MIMETYPE_VIDEO_DIVX, "OMX.TCC.divxdec" },
{ MEDIA_MIMETYPE_VIDEO_MPEG2, "OMX.TCC.mpeg2dec" },
{ MEDIA_MIMETYPE_VIDEO_MJPEG, "OMX.TCC.mjpegdec" },
{ MEDIA_MIMETYPE_VIDEO_FLV1, "OMX.TCC.flv1dec" },
}; 可以看到MPEG4就对应了6种Codec.
InstantiateSoftwareCodec创建软解码器
InstantiateSoftwareCodec从 kFactoryInfo (软解码器列表)挑挑看有没有. 有的话就创建一个软解码器.看看kFactoryInfo里面有哪些软解码器
static const FactoryInfo kFactoryInfo[] = {
FACTORY_REF(MP3Decoder)
FACTORY_REF(AMRNBDecoder)
FACTORY_REF(AMRWBDecoder)
FACTORY_REF(AACDecoder)
FACTORY_REF(AVCDecoder)
FACTORY_REF(G711Decoder)
FACTORY_REF(M4vH263Decoder)
FACTORY_REF(VorbisDecoder)
FACTORY_REF(VPXDecoder)
}; 解码器名称的一点说明
OMX.XXX.YYY 中间的XXX是厂商名称.如OMX.TI.Video.Decoder 就是TI芯片的硬视频解码器. 而 OMX.TCC.avcdec则是TCC的AVC视频解码器. 没有OMX开头的,说明是软解码器.
以AVC为例:
{ MEDIA_MIMETYPE_VIDEO_AVC, "OMX.qcom.7x30.video.decoder.avc" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "OMX.qcom.video.decoder.avc" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "OMX.TI.Video.Decoder" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "OMX.SEC.AVC.Decoder" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "OMX.TCC.avcdec" },
{ MEDIA_MIMETYPE_VIDEO_AVC, "AVCDecoder" },可以看到软解码器被放到最后.这样的话后面尝试申请解码器的时候便会优先申请硬Codec. 除非硬Codec不存在.
创建OMXCodec
申请OMXCodec比较简单,调用IOMX::allocateNode 申请即可.编解码器的名称例如 OMX.TCC.avcdec 即是OMX组件(Component)的名称
IOMX::node_id node = 0;
omx->allocateNode(componentName, observer, &node); 这个时候就已经是和OMX IL 层进行通讯了,虽然是进程间通讯. 但是封装成这个样子,我们也看不出来了,和本地调用一样.
sp<OMXCodec> codec = new OMXCodec(
omx, node, quirks,
createEncoder, mime, componentName,
source);
codec->configureCodec(meta, flags); // codec 创建出来后, 要配置一下codec.如果进去看看configureCodec的代码,可以看到实际上是调用 IOMX::setParameter, 和IOMX::setConfig. 同样,也是属于IPC,因为是和OMX IL 通讯.
总结:理想的情况下, 调用OMXCodec::Create应该返回一个OMXCodec 对象而不是软解对象. Android默认的策略也是优先创建硬解码器. 至此AwesomePlayer通过OMXCodec 进而跟OML IL打交道. 其中关键的对象为IOMX和IOMX::node_id. node_id相当于一个OMX Component的句柄.音频解码器的创建过程跟视频解码器的创建过程几乎一样, 所以不分析了.
⑥解封装, 解码
看之前的我们知道，当VideoEvent 被触发时, AwesomePlayer::onVideoEvent会被调用. onVideoEvent 会尝试调用 mVideoSource.read读取视频图像,然后将视频图像交给AwesomeRenderer进行渲染.
如果采用硬解码的话 mVideoSource实际是就是一个OMXCodec 对象.
至此，一个AwesomePlayer播放的实现就完成了！