The correct approach to the Rockit multimedia player audio decoding buffer is to allocate it in real time, because the audio decoding buffer size may not be fixed in some streams. Moreover, some streams cannot know the size of the decoding frame during initialization. If it is pre-allocated in advance, it is likely to cause a waste of buffer or the allocated buffer is too small, causing sound interruption. Therefore, it needs to be allocated in real time according to the decoding frame. However, the allocation method is created by plug-ins, so only plug-ins have the right to allocate and release, and the specific decoding
Rockit audio code simple logic line:
Buffer listening base class RTBufferListener:
class RTBufferListener {
public:
RTBufferListener() {}
virtual ~RTBufferListener() {}
public:
// buffer callback.
/*
* section: override virtual methods of RTBufferListener.
* name: onBufferAvailable
*/
virtual void onBufferAvailable(void* buffer) = 0;
/*
* section: override virtual methods of RTBufferListener.
* name: onBufferRealloc
*/
virtual void onBufferRealloc(void* buffer, UINT32 size) = 0;
/*
* section: override virtual methods of RTBufferListener.
* name: onBufferRelease
*/
virtual void onBufferRelease(void* buffer, RT_BOOL render) = 0;
};
What these three functions do:
void onBufferAvailable(void* buffer): Available buffer callback function
void onBufferRealloc(void* buffer, UINT32 size): buffer reallocation callback function
void onBufferRelease(void* buffer, RT_BOOL render): buffer release callback function
The audio decoding plug-in RTNodeAudioCodec and the buffer management pool RTMediaBufferPool respectively inherit this listening base class:
class RTNodeAudioCodec : public RTNodeCodec, RTBufferListener {
public:
RTNodeAudioCodec();
virtual ~RTNodeAudioCodec();
......
}
class RTMediaBufferPool : public RTBufferListener {
public:
/* create for external buffer pool */
explicit RTMediaBufferPool(UINT32 max_buffer_count = 0);
/* create for internal buffer pool */
explicit RTMediaBufferPool(UINT32 max_buffer_count, UINT32 buffer_size);
~RTMediaBufferPool();
......
}RTNodeAudioCodec will create new RTMediaBufferPool in initialization init(), and register itself to RTMediaBufferPool through setBufferListener(this)
RTNodeAudoCodec will get the allocation method (using malloc allocation method) in the prepare phase, and pre-allocate the buffer:
RT_RET RTNodeAudioCodec::onPrepare() {
// create malloc allocator for audio
RTAllocatorStore::fetchAllocator(RTAllocatorStore::RT_ALLOC_TYPE_MALLOC, mMetaInput, &mLinearAllocator);
RT_LOGD_IF(DEBUG_FLAG, "fetchAllocator(ALLOC_TYPE_MALLOC), mLinearAllocator = %p", mLinearAllocator);
RT_ASSERT(RT_NULL != mLinearAllocator);
// allocate input and output buffers.
allocateBuffersOnPort(RT_PORT_INPUT);
allocateBuffersOnPort(RT_PORT_OUTPUT);
mFramePool->start();
mPacketPool->start();
return RT_OK;
}Buffer pre-allocation. Here, the decoding rotation buffer is allocated empty (only the new RTMediaBuffer is rotated, and no actual memory is allocated), and the buffer is registered into the memory pool:
In RTMediaBufferPool->registerBuffer, RTMediaBufferPool will be registered to RTMediaBuffer:
After starting playback, RTNodeAudioCodec->runtask->IAudioCodec->fa_xx_get_frame will start monitoring the buffer:
When buffer->getSize() is smaller than the size of the decoded frame or buffer->getData() is empty, the buffer will be reallocated:
RT_RET fa_xx_get_frame(FABSEncoderContext* ctx, RTMediaBuffer *buffer) {
UINT8 *dst = RT_NULL;
if (ctx->mSize > 0) {
// reallocate MediaBuffer is if resample_size > buffer_size
if (buffer->getSize() < ctx->mSize) {
RT_LOGD_IF(DEBUG_FLAG, "reallocate buffer(this=%p, size=%d), old=%d", \
buffer, ctx->mSize, buffer->getSize());
}
buffer->signalBufferRealloc(ctx->mSize);
if (buffer->getSize() < ctx->mSize) {
RT_LOGD("fail to reallocate buffer(this=%p, size=%d) is smaller than ctx->mSize=%d", \
buffer, buffer->getSize(), ctx->mSize);
ctx->mSize = buffer->getSize();
}
......
}
......
}
Here buffer->signalBufferRealloc(ctx->mSize); will call:
void RTMediaBuffer::signalBufferRealloc(UINT32 size) {
if (RT_NULL != mBufferListener) {
mBufferListener->onBufferRealloc(this, size);
}
}The mBufferListener here is registered in the screenshot part of RTMediaBufferPool->registerBuffer above, so here it is called back to onBufferRealloc in RTMediaBufferPool:
/*
* section: override virtual methods of RTBufferListener.
* name: onBufferRealloc
*/
void RTMediaBufferPool::onBufferRealloc(void* buffer, UINT32 size) {
RT_LOGD_IF(DEBUG_FLAG, "buffer(%p) is reallocated", buffer);
if (RT_NULL != mBufferListener) {
mBufferListener->onBufferRealloc(buffer, size);
if (RT_NULL != buffer) {
RTMediaBuffer* frame = reinterpret_cast<RTMediaBuffer*>(buffer);
// need to increase the reference count when RTMediaBuffer is realloced,
// otherwise the RTMediaBuffer will be probabilistically recycled and reused,
// and objects that are using RTMediaBuffer will crash.
if (0 == frame->refsCount()) {
frame->addRefs();
}
// buffer pool manages this new buffer, so needs RTBufferListener.
frame->setListener(this);
}
}
}The mBufferListener in RTMediaBufferPool is registered by RTNodeAudioCodec, so RTNodeAudioCodec->onBufferRealloc is called here:
/*
* section: override virtual methods of RTBufferListener.
* name: onBufferRealloc
*/
void RTNodeAudioCodec::onBufferRealloc(void* buffer, UINT32 size) {
RTMediaBuffer* frame = reinterpret_cast<RTMediaBuffer*>(buffer);
if ((RT_NULL != frame) && (RT_NULL != mLinearAllocator)) {
if ((frame->getSize() < size) || (RT_NULL == frame->getData())) {
frame->release();
mLinearAllocator->freeBuffer(&frame);
mLinearAllocator->newBuffer(size, &frame);
RT_LOGD_IF(DEBUG_FLAG, "reallocated buffer(this: %p, data: %p, size: %d)",
frame, frame->getData(), size);
return;
}
}
}In this way, it will be judged whether to redistribute based on the distribution method registered by RTNodeAudioCodec, and the monitoring of who registers the distribution method and who manages the monitoring is realized.
Note: When the current buffer is reallocated, the buffer needs to be set with a reference mark (see screenshot box 1 below), because there is a probability that the buffer will be recycled or reused, causing a crash. And the buffer pool manages the newly allocated buffer, so Newly allocated ones need to be registered for monitoring (screenshot box 2 below).
