若有需要请查看前面章节分析：
【一】Android MediaRecorder整体架构源码浅析
【二】Android MediaRecorder C++底层架构音视频处理过程和音视频同步源码分析
然后分析音频数据源AudioSource的start()方法：
status_t AudioSource::start(MetaData *params) {
// 加锁多线程操作
Mutex::Autolock autoLock(mLock);
if (mStarted) {
return UNKNOWN_ERROR;
}
if (mInitCheck != OK) {
return NO_INIT;
}
// 默认的初始化变量
mTrackMaxAmplitude = false;
mMaxAmplitude = 0; // 最大振幅
mInitialReadTimeUs = 0;// 初始化真实时间
mStartTimeUs = 0;// 开始时间戳
int64_t startTimeUs; // 获取音视频共同的同步获取数据的时间戳
if (params && params->findInt64(kKeyTime, &startTimeUs)) {
mStartTimeUs = startTimeUs;
}
// 此处开始音频录制，下面进行分析
status_t err = mRecord->start();
if (err == OK) {
mStarted = true;
} else {
mRecord.clear();
}
return err;
}
分析mRecord->start();如下：
先分析下mRecord的来历，在构造方法AudioSource::AudioSource()中初始化的：
mRecord = new AudioRecord(
inputSource, sampleRate, AUDIO_FORMAT_PCM_16_BIT,
audio_channel_in_mask_from_count(channelCount),
opPackageName,
(size_t) (bufCount * frameCount),
AudioRecordCallbackFunction,
this,
frameCount /*notificationFrames*/,
AUDIO_SESSION_ALLOCATE,
AudioRecord::TRANSFER_DEFAULT,
AUDIO_INPUT_FLAG_NONE,
uid,
pid);
mInitCheck = mRecord->initCheck();
如上进行了创建AudioRecord实例和赋值，因此该分析AudioRecord的start方法:
status_t AudioRecord::start(AudioSystem::sync_event_t event, audio_session_t triggerSession)
{
ALOGV("start, sync event %d trigger session %d", event, triggerSession);
SEEMPLOG_RECORD(71,"");
AutoMutex lock(mLock);
if (mActive) {
return NO_ERROR;
}
/**
由void AudioRecord::getAudioService() 方法可知道：该对象是通过Binder机制从ServiceManager管理的系统服务中获取到的一个AudioService的Bp代理对象，用于和AudioService服务器端进行音频数据交互等：源码如：
void AudioRecord::getAudioService() {
if (mAudioService == NULL) {
const sp<IServiceManager> sm(defaultServiceManager());
if (sm != NULL) {
const String16 name("audio");
mAudioService = interface_cast<IAudioService>(sm->getService(name));
if (mAudioService == NULL) {
ALOGE("AudioService is NULL");
}else {
ALOGI("AudioService is NOT NULL");
}
}
mAudioStateController = new AudioStateController(this);
}
}
并且getAudioService()该方法是在构造函数中进行调用的，因此此处不为空
**/
if (mAudioService != NULL) {
ALOGI("call onAudioRecordStart when start");
// 通知远程的服务器端的Audio开始录制，并且给一个【mAudioStateController】变量进行操控客户端Audio音频录制交互
mAudioService->onAudioRecordStart(mClientUid, mAudioStateController);
}
/** 刷新数据即可能会丢弃此前存在的数据。
此处分析下mProxy变量的来历：在AudioRecord::openRecord_l()该方法中进行初始化的：
mProxy = new AudioRecordClientProxy(cblk, buffers, mFrameCount, mFrameSize);
即它是一个代理音频录制的客户端代理对象，代理的是服务器端的AudioRecord。
【AudioRecord::openRecord_l()】方法中会获取AudioFlinger的Bp代理服务对象，然后调用其audioFlinger->openRecord()打开录制音频数据的方法开始录制并且返回服务端AudioRecord的代理对象即sp<IAudioRecord> record最后赋值给mAudioRecord变量，
然后最重要的是获取到音频数据的共享内存块【audio_track_cblk_t】结构（类）实例进行操作共享音频数据内存。缓冲区要么紧接在控制块之后，
或在单独的区域由服务器端AudioService自行决定。
并且【AudioRecord::openRecord_l()】该方法在构造函数执行时就初始化执行了
**/
// discard data in buffer
const uint32_t framesFlushed = mProxy->flush();
mFramesReadServerOffset -= mFramesRead + framesFlushed;
mFramesRead = 0;
mProxy->clearTimestamp(); // timestamp is invalid until next server push
// reset current position as seen by client to 0
mProxy->setEpoch(mProxy->getEpoch() - mProxy->getPosition());
// 调用processAudioBuffer()时强制刷新剩余帧数据，并且在停止前读取可能是不完整的部分数据
// force refresh of remaining frames by processAudioBuffer() as last
// read before stop could be partial.
mRefreshRemaining = true;
mNewPosition = mProxy->getPosition() + mUpdatePeriod;
int32_t flags = android_atomic_acquire_load(&mCblk->mFlags);
// we reactivate markers (mMarkerPosition != 0) as the position is reset to 0.
// This is legacy behavior. This is not done in stop() to avoid a race condition
// where the last marker event is issued twice.
mMarkerReached = false;
mActive = true;
status_t status = NO_ERROR;
if (!(flags & CBLK_INVALID)) {
ALOGV("mAudioRecord->start()");
// ！！！此处开启了AudioFlinger服务器端的AudioRecord开始录制，此处不再深入AudioFlinger，以后有时间在进行
status = mAudioRecord->start(event, triggerSession);
if (status == DEAD_OBJECT) {
flags |= CBLK_INVALID;
}
}
if (flags & CBLK_INVALID) {
status = restoreRecord_l("start");
}
if (status != NO_ERROR) {
mActive = false;
ALOGE("start() status %d", status);
} else {
sp<AudioRecordThread> t = mAudioRecordThread;
if (t != 0) {
// 正常执行此处即mAudioRecordThread的resume方法，该方法作用是：
允许线程执行，如果没有要求则退出
t->resume();
} else {
mPreviousPriority = getpriority(PRIO_PROCESS, 0);
get_sched_policy(0, &mPreviousSchedulingGroup);
androidSetThreadPriority(0, ANDROID_PRIORITY_AUDIO);
}
}
return status;
}
接着分析：mAudioRecordThread的resume方法：
先分析mAudioRecordThread的来历：即AudioRecord的构造函数会执行如下语句：
if (cbf != NULL) {
mAudioRecordThread = new AudioRecordThread(*this, threadCanCallJava);
mAudioRecordThread->run("AudioRecord", ANDROID_PRIORITY_AUDIO);
// 现在在暂停状态就开始了
// thread begins in paused state, and will not reference us until start()
}
【cbf】是一个回调函数指针，即来自于如下：（在AudioSource.cpp中定义的）
static void AudioRecordCallbackFunction(int event, void *user, void *info) {
AudioSource *source = (AudioSource *) user;
switch (event) {
case AudioRecord::EVENT_MORE_DATA: {
source->dataCallback(*((AudioRecord::Buffer *) info));
break;
}
case AudioRecord::EVENT_OVERRUN: {
ALOGW("AudioRecord reported overrun!");
break;
}
default:
// does nothing
break;
}
}
此函数是非常重要的，用于回调接收AudioFlinger服务端层音频数据回调的，此处先不分析。
再看进行了new AudioRecordThread(*this, threadCanCallJava);操作，创建了其对象实例，
并将客服端的AudioRecord实例作为数据接收者来传入，用于在新线程中通知客户端。
因此在看mAudioRecordThread的resume方法方法时即如下：
void AudioRecord::AudioRecordThread::resume()
{
AutoMutex _l(mMyLock);
mIgnoreNextPausedInt = true;
if (mPaused || mPausedInt) {
mPaused = false;
mPausedInt = false;
/** 此处非常作用的一个信号量机制条件锁来控制其重新线程开启线程循环loop来获取音频数据，该信号量唤醒是在线程循环threadLoop()函数中
**/
mMyCond.signal();
}
}
接着分析：线程循环threadLoop()函数
bool AudioRecord::AudioRecordThread::threadLoop()
{
{
AutoMutex _l(mMyLock);
if (mPaused) {
// TODO check return value and handle or log
// 信号量等待被唤起该操作
mMyCond.wait(mMyLock);
// caller will check for exitPending()
return true;
}
if (mIgnoreNextPausedInt) {
mIgnoreNextPausedInt = false;
mPausedInt = false;
}
if (mPausedInt) {
if (mPausedNs > 0) {
// TODO check return value and handle or log
(void) mMyCond.waitRelative(mMyLock, mPausedNs);
} else {
// 信号量等待被唤起该操作
// TODO check return value and handle or log
mMyCond.wait(mMyLock);
}
mPausedInt = false;
return true;
}
}
if (exitPending()) {
return false;
}
// 执行了AudioRecord的processAudioBuffer()该方法
nsecs_t ns = mReceiver.processAudioBuffer();
switch (ns) {
case 0:
return true;
case NS_INACTIVE:
pauseInternal();
return true;
case NS_NEVER:
return false;
case NS_WHENEVER:
// Event driven: call wake() when callback notifications conditions change.
ns = INT64_MAX;
// fall through
default:
LOG_ALWAYS_FATAL_IF(ns < 0, "processAudioBuffer() returned %" PRId64, ns);
pauseInternal(ns);
return true;
}
}
因此可以看出音频内部是通在一个新线程中开启线程循环获取数据操作来达到数据读取和写入的同步进行，并且读取是通过processAudioBuffer()该方法不断循环loop
再分析AudioRecord的processAudioBuffer()该方法：
该方法非常多的处理，主要是校验和读取音频帧数据后进行封装数据后回调等操作，关键代码如下：
size_t readFrames = 0;
/** 获取到的帧数据大小是在AudioRecord::obtainBuffer()方法中获取到的，该方法是其AudioRecord::read()方法执行时进行调用获取音频数据的
**/
while (mRemainingFrames > 0) {
/**
音频数据被读取后封装后的数据结构为Buffer类型
**/
Buffer audioBuffer;
audioBuffer.frameCount = mRemainingFrames;
size_t nonContig;
// 然后在AudioRecord:: obtainBuffer该方法中循环获取每一帧的音频数据并将其打包成Buffer类型数据，后面分析该方法
status_t err = obtainBuffer(&audioBuffer, requested, NULL, &nonContig);
LOG_ALWAYS_FATAL_IF((err != NO_ERROR) != (audioBuffer.frameCount == 0),
"obtainBuffer() err=%d frameCount=%zu", err, audioBuffer.frameCount);
requested = &ClientProxy::kNonBlocking;
// 记录当前帧buffer中包含的数据帧数个数
size_t avail = audioBuffer.frameCount + nonContig;
ALOGV("obtainBuffer(%u) returned %zu = %zu + %zu err %d",
mRemainingFrames, avail, audioBuffer.frameCount, nonContig, err);
if (err != NO_ERROR) {
if (err == TIMED_OUT || err == WOULD_BLOCK || err == -EINTR) {
break;
}
ALOGE("Error %d obtaining an audio buffer, giving up.", err);
return NS_NEVER;
}
if (mRetryOnPartialBuffer) {
mRetryOnPartialBuffer = false;
if (avail < mRemainingFrames) {
int64_t myns = ((mRemainingFrames - avail) *
1100000000LL) / mSampleRate;
if (ns < 0 || myns < ns) {
ns = myns;
}
return ns;
}
}
// 记录当前帧音频buffer大小
size_t reqSize = audioBuffer.size;
// 进行回调给上层，并将数据对象地址传递回去，因此可以知道音频数据内存是在Audio
mCbf(EVENT_MORE_DATA, mUserData, &audioBuffer);
// callback上层已经读取的数据大小，用于比较是否读取数据正常
size_t readSize = audioBuffer.size;
// 比较是否读取数据正常
// Sanity check on returned size
if (ssize_t(readSize) < 0 || readSize > reqSize) {
ALOGE("EVENT_MORE_DATA requested %zu bytes but callback returned %zd bytes",
reqSize, ssize_t(readSize));
return NS_NEVER;
}
if (readSize == 0) {
// 等于0的情况看英文注释就懂了，就是上层还在处理消耗buffer数据，因此此处让线程wait即等待10秒间隔
// The callback is done consuming buffers
// Keep this thread going to handle timed events and
// still try to provide more data in intervals of WAIT_PERIOD_MS
// but don't just loop and block the CPU, so wait
return WAIT_PERIOD_MS * 1000000LL;
}
/** 这里处理已经被读取后的数据应该要被释放掉帧数据内存，并且减少可读取帧个数即mRemainingFrames
**/
size_t releasedFrames = readSize / mFrameSize;
audioBuffer.frameCount = releasedFrames;
mRemainingFrames -= releasedFrames;
if (misalignment >= releasedFrames) {
misalignment -= releasedFrames;
} else {
misalignment = 0;
}
/** 释放当前音频buffer，此处进行分析源码如下：
void AudioRecord::releaseBuffer(const Buffer* audioBuffer)
{
// FIXME add error checking on mode, by adding an internal version
size_t stepCount = audioBuffer->size / mFrameSize;
if (stepCount == 0) {
return;
}
Proxy::Buffer buffer;
buffer.mFrameCount = stepCount;
buffer.mRaw = audioBuffer->raw;
AutoMutex lock(mLock);
mInOverrun = false;
mProxy->releaseBuffer(&buffer);
// the server does not automatically disable recorder on overrun, so no need to restart
}
即将AudioRecord的Buffer结构数据封装成mProxy代理对象保存的Proxy::Buffer数据类型，使其能够找到对应buffer进行释放，Proxy::Buffer结构如下：
struct Buffer {
size_t mFrameCount; // number of frames available in this buffer
void* mRaw; // pointer to first frame
size_t mNonContig; // number of additional non-contiguous frames available
};
而【mProxy->releaseBuffer(&buffer);】最终会调用：
ClientProxy::releaseBuffer(Buffer* buffer)方法然后执行如下释放内存：
size_t stepCount = buffer->mFrameCount;
if (stepCount == 0 || mIsShutdown) {
// prevent accidental re-use of buffer
buffer->mFrameCount = 0;
buffer->mRaw = NULL;
buffer->mNonContig = 0;
return;
}
**/
releaseBuffer(&audioBuffer);
// 记录当前已经读取帧数据的个数大小
readFrames += releasedFrames;
// FIXME here is where we would repeat EVENT_MORE_DATA again on same advanced buffer
// if callback doesn't like to accept the full chunk
if (readSize < reqSize) {
continue;
}
// There could be enough non-contiguous frames available to satisfy the remaining request
if (mRemainingFrames <= nonContig) {
continue;
}
#if 0
// This heuristic tries to collapse a series of EVENT_MORE_DATA that would total to a
// sum <= notificationFrames. It replaces that series by at most two EVENT_MORE_DATA
// that total to a sum == notificationFrames.
if (0 < misalignment && misalignment <= mRemainingFrames) {
mRemainingFrames = misalignment;
return (mRemainingFrames * 1100000000LL) / mSampleRate;
}
#endif
}
if (readFrames > 0) {
AutoMutex lock(mLock);
// 然后记录当前已读取的帧音频数据个数
mFramesRead += readFrames;
// mFramesReadTime = systemTime(SYSTEM_TIME_MONOTONIC); // not provided at this time.
}
mRemainingFrames = notificationFrames;
mRetryOnPartialBuffer = true;
上面分析到音频数据结构被封装在Buffer类结构数据，定义如下：
/* Client should declare a Buffer and pass address to obtainBuffer()
* and releaseBuffer(). See also callback_t for EVENT_MORE_DATA.
*/
class Buffer
{
public:
// FIXME use m prefix
size_t frameCount; // number of sample frames corresponding to size;
// on input to obtainBuffer() it is the number of frames desired
// on output from obtainBuffer() it is the number of available
// frames to be read
// on input to releaseBuffer() it is currently ignored
size_t size; // input/output in bytes == frameCount * frameSize
// on input to obtainBuffer() it is ignored
// on output from obtainBuffer() it is the number of available
// bytes to be read, which is frameCount * frameSize
// on input to releaseBuffer() it is the number of bytes to
// release
// FIXME This is redundant with respect to frameCount. Consider
// removing size and making frameCount the primary field.
union {
void* raw;
short* i16; // signed 16-bit
int8_t* i8; // unsigned 8-bit, offset by 0x80
// input to obtainBuffer(): unused, output: pointer to buffer
};
};
接着分析：AudioRecord:: obtainBuffer() 方法，该方法会去获取音频帧数据：
status_t AudioRecord::obtainBuffer(Buffer* audioBuffer, const struct timespec *requested,
struct timespec *elapsed, size_t *nonContig)
{
// 新旧序列数据个数
// previous and new IAudioRecord sequence numbers are used to detect track re-creation
uint32_t oldSequence = 0;
uint32_t newSequence;
Proxy::Buffer buffer;
status_t status = NO_ERROR;
static const int32_t kMaxTries = 5;
int32_t tryCounter = kMaxTries;
do {
// obtainBuffer() is called with mutex unlocked, so keep extra references to these fields to
// keep them from going away if another thread re-creates the track during obtainBuffer()
sp<AudioRecordClientProxy> proxy;
sp<IMemory> iMem;
sp<IMemory> bufferMem;
{
// start of lock scope
AutoMutex lock(mLock);
newSequence = mSequence;
// 此前audioservice中断等异常无效时重新打开start连接进行录制
// did previous obtainBuffer() fail due to media server death or voluntary invalidation?
if (status == DEAD_OBJECT) {
// re-create track, unless someone else has already done so
if (newSequence == oldSequence) {
status = restoreRecord_l("obtainBuffer");
if (status != NO_ERROR) {
buffer.mFrameCount = 0;
buffer.mRaw = NULL;
buffer.mNonContig = 0;
break;
}
}
}
oldSequence = newSequence;
// Keep the extra references
proxy = mProxy;
iMem = mCblkMemory; // audioFlinger的callback
bufferMem = mBufferMemory;
// Non-blocking if track is stopped
if (!mActive) {
requested = &ClientProxy::kNonBlocking;
}
} // end of lock scope
/** 此处转换成mProxy代理AudioRecordProxy实例的数据结构即Proxy::Buffer buffer;。
然后通过代理mProxy实例进行获取obtainBuffer()，下面会继续分析
**/
buffer.mFrameCount = audioBuffer->frameCount;
// FIXME starts the requested timeout and elapsed over from scratch
status = proxy->obtainBuffer(&buffer, requested, elapsed);
} while ((status == DEAD_OBJECT) && (tryCounter-- > 0));
// 获取到之后然后将Proxy::Buffer结构数据封装成Buffer结构，返回回去
audioBuffer->frameCount = buffer.mFrameCount;
audioBuffer->size = buffer.mFrameCount * mFrameSize;
// 这是原始音频数据，指向第一帧数据内存的指针，使用的是共享内存
audioBuffer->raw = buffer.mRaw;
if (nonContig != NULL) {
*nonContig = buffer.mNonContig;
}
return status;
}
分析：proxy->obtainBuffer(&buffer, requested, elapsed);如下：
最终调用了：
status_t ClientProxy::obtainBuffer(Buffer* buffer, const struct timespec *requested,
struct timespec *elapsed)方法，该方法非常多的处理，此处关键处理如下：
开启一个无条件for循环：
for (;;) {
// compute number of frames available to write (AudioTrack) or read (AudioRecord)
int32_t front;
int32_t rear;
// 从共享内存区域获取帧数据前后中间增加的数据，即本次被填入共享内存区域的数据
if (mIsOut) {
// The barrier following the read of mFront is probably redundant.
// We're about to perform a conditional branch based on 'filled',
// which will force the processor to observe the read of mFront
// prior to allowing data writes starting at mRaw.
// However, the processor may support speculative execution,
// and be unable to undo speculative writes into shared memory.
// The barrier will prevent such speculative execution.
front = android_atomic_acquire_load(&cblk->u.mStreaming.mFront);
rear = cblk->u.mStreaming.mRear;
} else {
// On the other hand, this barrier is required.
rear = android_atomic_acquire_load(&cblk->u.mStreaming.mRear);
front = cblk->u.mStreaming.mFront;
}
// 从后面写入，从前面读取，中间就是数据部分
// write to rear, read from front
ssize_t filled = rear - front;
// 管道不应该太满
// pipe should not be overfull
if (!(0 <= filled && (size_t) filled <= mFrameCount)) {
if (mIsOut) {
ALOGE("Shared memory control block is corrupt (filled=%zd, mFrameCount=%zu); "
"shutting down", filled, mFrameCount);
mIsShutdown = true;
status = NO_INIT;
goto end;
}
// for input, sync up on overrun
filled = 0;
cblk->u.mStreaming.mFront = rear;
(void) android_atomic_or(CBLK_OVERRUN, &cblk->mFlags);
}
// Don't allow filling pipe beyond the user settable size.
// The calculation for avail can go negative if the buffer size
// is suddenly dropped below the amount already in the buffer.
// So use a signed calculation to prevent a numeric overflow abort.
ssize_t adjustableSize = (ssize_t) getBufferSizeInFrames();
ssize_t avail = (mIsOut) ? adjustableSize - filled : filled;
if (avail < 0) {
avail = 0;
} else if (avail > 0) {
// “avail”可能是非连续的，因此只返回第一个连续块
// 'avail' may be non-contiguous, so return only the first contiguous chunk
size_t part1;
if (mIsOut) {
rear &= mFrameCountP2 - 1;
part1 = mFrameCountP2 - rear;
} else {
front &= mFrameCountP2 - 1;
part1 = mFrameCountP2 - front;
}
if (part1 > (size_t)avail) {
part1 = avail;
}
if (part1 > buffer->mFrameCount) {
part1 = buffer->mFrameCount;
}
// 进行Buffer数据读取赋值
buffer->mFrameCount = part1;
buffer->mRaw = part1 > 0 ?
&((char *) mBuffers)[(mIsOut ? rear : front) * mFrameSize] : NULL;
buffer->mNonContig = avail - part1;
mUnreleased = part1;
status = NO_ERROR;
break;
}
// 。。。。省略部分代码
// ！！！此处代码暂不能分析，其大概作用一个是系统调用共享内存区域内的状态变成等待唤醒吧，是一种同步操作？？
(void) syscall(__NR_futex, &cblk->mFutex,
mClientInServer ? FUTEX_WAIT_PRIVATE : FUTEX_WAIT, old & ~CBLK_FUTEX_WAKE, ts);
status_t error = errno; // clock_gettime can affect errno
// update total elapsed time spent waiting
}
如此也就将共享内存中的数据获取到了其指定的数据地址，然后上层进行数据指针读取处理。
再次分析此前未完成分析的Puller部分
分析【schedulePull()】方法实际是发送了【kWhatPull】该事件：
void MediaCodecSource::Puller::onMessageReceived(const sp &msg)该方法中处理的：
case kWhatPull:
{
Mutexed<Queue>::Locked queue(mQueue);
// 记录当前队列添加数据的时间戳
queue->mReadPendingSince = ALooper::GetNowUs();
if (!queue->mPulling) {
handleEOS();
break;
}
queue.unlock();
// 读取到的MediaBuffer数据结构，读取到的数据将放入此缓存数据对象中
MediaBuffer *mbuf = NULL;
/** 读取数据操作，此处也是非常重要的即调用了CameraSource或者AudioSource的read()函数读取未编码的原始音视频数据，拉取回来后让MediaCodecSource即encoder进行编码操作，下面会进行详细分析
**/
status_t err = mSource->read(&mbuf);
queue.lock();
queue->mReadPendingSince = 0;
/** 获取数据后，此处做了是否需要丢弃数据源，比如还没有开始拉取或暂停时就应该丢弃获取到的当前MediaBuffer数据，然后释放其数据内存
**/
// if we need to discard buffer
if (!queue->mPulling || queue->mPaused || err != OK) {
if (mbuf != NULL) {
mbuf->release();
mbuf = NULL;
}
if (queue->mPulling && err == OK) {
// 当暂停时，也要求继续读取数据，即此处是继续执行发送【kWhatPull】该事件：
原因是该msg的Handler本来就是Puller本身。
msg->post(); // if simply paused, keep pulling source
break;
} else if (err == ERROR_END_OF_STREAM) {
ALOGV("stream ended, mbuf %p", mbuf);
} else if (err != OK) {
ALOGE("error %d reading stream.", err);
}
}
// 读取到了音视频原始数据后，将其添加push到队列Buffer缓冲区中
if (mbuf != NULL) {
queue->pushBuffer(mbuf);
}
queue.unlock();
if (mbuf != NULL) {
/** 然后通知mNotify即之前的携带有【kWhatPullerNotify】事件通知发送给MediaCodecSource的onMessageReceived()方法去接收已经获取到的原始音视频数据，进行编码。下面会继续分析该方法的处理流程的
**/
mNotify->post();
// 此句上面已经分析了即是继续读取数据，即此处是继续执行发送【kWhatPull】该事件：
原因是该msg的Handler本来就是Puller本身。也就是拉取数据和编码数据是异步进行的。
msg->post();
} else {
handleEOS();
}
break;
}
分析：CameraSource或者AudioSource的read方法：
处理的要点：读取和写入数据的同步条件信号量机制来进行的，但有一点是非常重要的，就是：底层Driver层回调写入的数据是不会被停止的，因为是回调触发的，数据都会放入对应的缓冲集合中，只有读取的时候为空时才会停止进行等待并且有等待超时处理其实还是又继续等待了，除非状态不是start状态了就退出了。
首先分析CameraSource的read方法：
分析前看几个非常重要的变量：
Mutex mLock;
// 读取和写入数据的同步条件信号量机制
Condition mFrameAvailableCondition;
Condition mFrameCompleteCondition;
// 从Driver层读取接收到的帧数据聚合
List<sp<IMemory> > mFramesReceived;
// 将要可以被编码的帧数据集合，注意是未编码的数据将要被编码
List<sp<IMemory> > mFramesBeingEncoded;
// 帧数据集合对应的帧时间戳集合
List<int64_t> mFrameTimes;
status_t CameraSource::read(
MediaBuffer **buffer, const ReadOptions *options) {
ALOGV("read");
*buffer = NULL;
// 默认options参数是空的
int64_t seekTimeUs;
ReadOptions::SeekMode mode;
if (options && options->getSeekTo(&seekTimeUs, &mode)) {
return ERROR_UNSUPPORTED;
}
/** 从底层Driver层读取到的数据是通过IMemory的Binder机制来传递数据地址的，如此操作数据地址来读取对应大小的数据即可
**/
sp<IMemory> frame;
int64_t frameTime;
{
// 加锁处理
Mutex::Autolock autoLock(mLock);
// 下面会分析如何向mFramesReceived加入数据的
while (mStarted && mFramesReceived.empty()) {
// 刚开始的时候接收到的帧集合数据肯定为空，此时进行超时设置等待
if (NO_ERROR !=
mFrameAvailableCondition.waitRelative(mLock,
mTimeBetweenFrameCaptureUs * 1000LL + CAMERA_SOURCE_TIMEOUT_NS)) {
if (mCameraRecordingProxy != 0 &&
!IInterface::asBinder(mCameraRecordingProxy)->isBinderAlive()) {
ALOGW("camera recording proxy is gone");
return ERROR_END_OF_STREAM;
}
// 超时等待视频帧数据传递回来时等待了多久
ALOGW("Timed out waiting for incoming camera video frames: %" PRId64 " us",
mLastFrameTimestampUs);
}
}
if (!mStarted) {
return OK;
}
// 获取第一帧的IMemory结构数据
frame = *mFramesReceived.begin();
// 从集合数据中移除该项数据
mFramesReceived.erase(mFramesReceived.begin());
// 读取响应的第一帧数据时间戳
frameTime = *mFrameTimes.begin();
// 从集合数据中移除该项数据
mFrameTimes.erase(mFrameTimes.begin());
// 然后将其接收到的视频帧数据放入待编码的数据集合中
mFramesBeingEncoded.push_back(frame);
/** 并且将该IMemory结构的数据封装成上层使用的MediaBuffer结构数据，但携带的数据都是相同的
**/
*buffer = new MediaBuffer(frame->pointer(), frame->size());
/** 此处也非常重要，将CameraSource的实例放入视频Buffer中返回给Puller，使其Puller读取完成后能够进行释放CameraSource这边分配的数据内存空间
**/
(*buffer)->setObserver(this);
// 引用计数加1，但它却不是智能指针哦，仅仅是MediaBuffer的一个引用计数int常量
(*buffer)->add_ref();
// 并且也设置当前帧数据对应的帧时间戳
(*buffer)->meta_data()->setInt64(kKeyTime, frameTime);
}
return OK;
}
分析如何向mFramesReceived加入数据的：
从此前我们的Camera部分分析知道，Camera开始后会回调如下三个方法：
CameraSource::dataCallbackTimestamp()；
CameraSource::recordingFrameHandleCallbackTimestamp();
CameraSource::recordingFrameHandleCallbackTimestampBatch();
这三个方法的原理都差不多，都是底层Driver通知回调对应的方法后，从中读取数据保存在mFramesReceived集合中。
首先分析：CameraSource::dataCallbackTimestamp()；
void CameraSource::dataCallbackTimestamp(int64_t timestampUs,
int32_t msgType __unused, const sp<IMemory> &data) {
ALOGV("dataCallbackTimestamp: timestamp %lld us", (long long)timestampUs);
Mutex::Autolock autoLock(mLock);
/** 是否应该跳过该帧数据，即主要是根据开始时间戳和获取到的数据时间戳进行比较，
若是开始时间大于收到的数据时间，那么代表着该帧数据是无效数据，因为还没有开始呢，然后将其是否数据内存，若是正确则调整开始的时间戳
**/
if (shouldSkipFrameLocked(timestampUs)) {
releaseOneRecordingFrame(data);
return;
}
// 记录接收到的视频帧数据个数加1
++mNumFramesReceived;
CHECK(data != NULL && data->size() > 0);
// 将其放入接收帧数据集合mFramesReceived中
mFramesReceived.push_back(data);
// 然后计算当前帧对应的时间戳，并将其存入时间戳集合中
int64_t timeUs = mStartTimeUs + (timestampUs - mFirstFrameTimeUs);
mFrameTimes.push_back(timeUs);
ALOGV("initial delay: %" PRId64 ", current time stamp: %" PRId64,
mStartTimeUs, timeUs);
/** 非常重要的信号量机制其重要了，通知此前read()方法执行时进行的等待操作被唤醒，使其能够继续进行读取数据。
**/
mFrameAvailableCondition.signal();
}
如此以上read和callback方法中间就形成了一种消费者和生产者的模式进行数据写入和读取，但是写入大小没有直接限制而已
然后分析AudioSource的read方法：
重要的变量：
// 接收音频原始数据的数据集合
List<MediaBuffer * > mBuffersReceived;
status_t AudioSource::read(
MediaBuffer **out, const ReadOptions * /* options */) {
Mutex::Autolock autoLock(mLock);
*out = NULL;
if (mInitCheck != OK) {
return NO_INIT;
}
// 与CameraSource处理基本一致，不过是没有超时机制的，其实原来结果都一样的
while (mStarted && mBuffersReceived.empty()) {
// 刚开始数据为空时肯定wait等待了
mFrameAvailableCondition.wait(mLock);
//当没有音频帧数据的时候直接结束返回，此时数据为空
if (mNoMoreFramesToRead) {
return OK;
}
}
if (!mStarted) {
return OK;
}
// 获取缓冲区集合中第一个数据
MediaBuffer *buffer = *mBuffersReceived.begin();
// 然后从集合中移除该数据
mBuffersReceived.erase(mBuffersReceived.begin());
// 记录客户端拥有的buffer数据个数
++mNumClientOwnedBuffers;
// 让客户端可操作release等
buffer->setObserver(this);
buffer->add_ref();
// 静音/抑制录音
// Mute/suppress the recording sound
int64_t timeUs;
CHECK(buffer->meta_data()->findInt64(kKeyTime, &timeUs));
/** 根据当前帧数据时间戳和当前的开始录制时间戳，获得此帧数据是否有效
即如果此帧数据早于开始录制时间戳，那么代表数据无效，不能要
**/
int64_t elapsedTimeUs = timeUs - mStartTimeUs;
if (elapsedTimeUs < kAutoRampStartUs) {
// 初始化数据为0，其长度默认也是0
memset((uint8_t *) buffer->data(), 0, buffer->range_length());
} else if (elapsedTimeUs < kAutoRampStartUs + kAutoRampDurationUs) {
int32_t autoRampDurationFrames =
((int64_t)kAutoRampDurationUs * mSampleRate + 500000LL) / 1000000LL; //Need type casting
int32_t autoRampStartFrames =
((int64_t)kAutoRampStartUs * mSampleRate + 500000LL) / 1000000LL; //Need type casting
int32_t nFrames = mNumFramesReceived - autoRampStartFrames;
// 渐变线性提高音量处理，从开始帧，到持续帧数
rampVolume(nFrames, autoRampDurationFrames,
(uint8_t *) buffer->data(), buffer->range_length());
}
// 跟踪最大音频录制信号振幅
// Track the max recording signal amplitude.
if (mTrackMaxAmplitude) {
trackMaxAmplitude(
(int16_t *) buffer->data(), buffer->range_length() >> 1);
}
// 帧数据的采样率若和输出目标采样率不一样时，重新设置其时间戳
if (mSampleRate != mOutSampleRate) {
timeUs *= (int64_t)mSampleRate / (int64_t)mOutSampleRate;
buffer->meta_data()->setInt64(kKeyTime, timeUs);
}
// 返回该buffer数据给上层
*out = buffer;
return OK;
}
分析音频数据如何添加进入mBuffersReceived的：
首先找到添加是在【AudioSource::queueInputBuffer_l】方法中
而该方法是在【AudioSource::dataCallback】即底层回调数据上传时调用的回调方法中进行调用的。
而【AudioSource::dataCallback】该方法是在如下回调函数中调用的：
static void AudioRecordCallbackFunction(int event, void *user, void *info) {
AudioSource *source = (AudioSource *) user;
switch (event) {
case AudioRecord::EVENT_MORE_DATA: {
source->dataCallback(*((AudioRecord::Buffer *) info));
break;
}
case AudioRecord::EVENT_OVERRUN: {
ALOGW("AudioRecord reported overrun!");
break;
}
default:
// does nothing
break;
}
}
而该方法AudioRecordCallbackFunction()此前的AudioSource分析时就已经提到过，它是底层AudioRecord获取到数据后进行回调上传数据给AudioSource的唯一路径。
因此我们可以直接分析【AudioSource::dataCallback】该方法了：
AudioSource::dataCallback(const AudioRecord::Buffer& audioBuffer)该方法处理挺多的，
只分析如下关键部分：
Mutex::Autolock autoLock(mLock);
if (!mStarted) {
ALOGW("Spurious callback from AudioRecord. Drop the audio data.");
return OK;
}
// 记录当前buffer的size
const size_t bufferSize = audioBuffer.size;
// 此处做丢弃此前已经检索到或此前已经丢失的音频数据
// Drop retrieved and previously lost audio data.
if (mNumFramesReceived == 0 && timeUs < mStartTimeUs) {
(void) mRecord->getInputFramesLost();
int64_t receievedFrames = bufferSize / mRecord->frameSize();
ALOGV("Drop audio data(%" PRId64 " frames) at %" PRId64 "/%" PRId64 " us",
receievedFrames, timeUs, mStartTimeUs);
// 记录丢失的帧数据个数
mNumFramesSkipped += receievedFrames;
return OK;
}
// 第一帧数据时做初始化处理
if (mNumFramesReceived == 0 && mPrevSampleTimeUs == 0) {
mInitialReadTimeUs = timeUs;
// Initial delay
if (mStartTimeUs > 0) {
mStartTimeUs = timeUs - mStartTimeUs;
}
mPrevSampleTimeUs = mStartTimeUs;
}
// 记录当前最新的帧数据时间戳
mLastFrameTimestampUs = timeUs;
// 会处理已检索到的或此前已丢失的音频数据
while (numLostBytes > 0) {
size_t bufferSize = numLostBytes;
if (numLostBytes > mMaxBufferSize) {
numLostBytes -= mMaxBufferSize;
bufferSize = mMaxBufferSize;
} else {
numLostBytes = 0;
}
// 若有丢失的数据，则也将其丢失的数据全部初始化0封装进MediaBuffer中并传递给上层进行处理
MediaBuffer *lostAudioBuffer = new MediaBuffer(bufferSize);
// 将数据进行全部初始化为空数据
memset(lostAudioBuffer->data(), 0, bufferSize);
lostAudioBuffer->set_range(0, bufferSize);
mNumFramesLost += bufferSize / mRecord->frameSize();
// 然后调用该方法，将buffer放入输入缓冲区中
// 这里把此前未处理完或丢失的帧数据继续使用空白数据进行填充
queueInputBuffer_l(lostAudioBuffer, timeUs);
}
// 有数据，将有效数据封装成上层接收的MediaBuffer
MediaBuffer *buffer = new MediaBuffer(bufferSize);
// 拷贝audioBuffer多个bytes的音频原始数据
memcpy((uint8_t *) buffer->data(),
audioBuffer.i16, audioBuffer.size);
// 记录当前有效数据的buffer大小
buffer->set_range(0, bufferSize);
// 然后调用该方法，将buffer放入输入缓冲区中
queueInputBuffer_l(buffer, timeUs);
分析：queueInputBuffer_l(buffer, timeUs);如下：
void AudioSource::queueInputBuffer_l(MediaBuffer *buffer, int64_t timeUs) {
// buffer 的大小即一个读取数据的范围长度
const size_t bufferSize = buffer->range_length();
// 帧数据大小
const size_t frameSize = mRecord->frameSize();
// 计算时间戳 微秒
const int64_t timestampUs =
mPrevSampleTimeUs +
((1000000LL * (bufferSize / frameSize)) +
(mSampleRate >> 1)) / mSampleRate;
if (mNumFramesReceived == 0) {
// 如果是第一帧数据，则设置一个锚定时间
buffer->meta_data()->setInt64(kKeyAnchorTime, mStartTimeUs);
}
// 设置此前帧数据的采样时间戳
buffer->meta_data()->setInt64(kKeyTime, mPrevSampleTimeUs);
// 设置其偏移时间戳
buffer->meta_data()->setInt64(kKeyDriftTime, timeUs - mInitialReadTimeUs);
// 重新记录成该帧数据的时间戳
mPrevSampleTimeUs = timestampUs;
// 计算本次帧数据的个数并累加记录缓存
mNumFramesReceived += bufferSize / frameSize;
// 将该缓存区数据放入集合中
mBuffersReceived.push_back(buffer);
// 通过信号量机制，通知read()方法可以读取帧数据了
mFrameAvailableCondition.signal();
}
如此以上也完成了集合数据的写入和读取，本质是通过信号量条件锁进行的，
基于消费者生产者模式。
上面已经分析完read处理流程了，接下来继续分析读取原始的音视频数据后又是如何进行编码和写入文件数据的呢，请看下文
请接着第四章的分析
【四】Android MediaRecorder C++底层架构音视频处理过程和音视频同步源码分析
MediaRecorder本系列章节内容分析已在一两年前分析完成的，当初只是用于分析的笔记记录下来，如今已走上了我向往的音视频开发领域，也调研和参与过音视频相关技术，目前喜爱C++底层音视频技术的开发，偏向于底层的音视频复用/解复用、解码/编码、推流拉流技术等，而当初分析本章内容时音视频技术积累较少，因此若本文中分析有误还请多多指教，谢谢。
过了这么久时间才分享出来的原因是，以往技术知识也向前辈们分享文章有所收获，现如今我也可以有所技术分享给需要的人，也希望可以帮忙到需要的人，技术分享帮助他人的同时也能作为自身技术掌握的总结记录。
以往忙于技术深入和研究未有所分享的技术，往后坚持有质量的技术分享。