AAudio is a new Android C API introduced in Android O. It is primarily designed for high-performance audio applications that require low latency. Applications communicate with AAudio by reading directly from or writing data to the stream, but it only contains basic audio input and output capabilities. Android's official document AAudio gives a good introduction to AAudio's API and design ideas. Here's a look at the implementation of AAudio. The following code analysis is based on android-12.1.0_r27.
AAudio moves audio data between applications and audio input and output on the Android device. The application passes in the audio stream and reads data from the audio stream to achieve this audio data transfer. The implementation of AAudio revolves around audio data transfer and audio streaming.
There are two main working modes for AAudio data transfer. One is the MMap mode. At this time, the client AAudio library media.aaudioobtains a shared memory from the service and transmits data to the device through this shared memory; the other is the so-called traditional mode. At this time, the interface of client AAudio is based on the traditional android::AudioTrackand implementation, using the same components android::AudioRecordas Java's AudioTrackand implementation. AudioRecordWhat can really achieve the performance and latency advantages described in the AAudio documentation is the working mode of mmap-based shared memory and device transfer data.
The implementation of the AAudio library mainly includes the following modules, and its source code directory structure is also organized according to modules:
- API interfaces, which directly implement the AAudio C interface called in the application, and the relevant code is located in
libaaudio/src/corethe directory; - legacy, an audio stream implemented based on the traditional
android::AudioTrackand interface, the relevant code is located in the directory;android::AudioRecordlibaaudio/src/legacy - Binding, client binding, is mainly used to
media.aaudiocommunicate with services, such asmedia.aaudiosending requests to or receivingmedia.aaudioevent notifications from services. The relevant code is located inlibaaudio/src/bindingthe directory; - fifo is mainly used for
media.aaudiodata transfer with services or devices. AAudio obtains the fd bound to the shared memory through the binding module. This fd needs to be mmapped again in the current process. The related classes of the fifo module are used to share the memory through these mmaps.media.aaudioFor data transfer with services or devices, the relevant code is located inlibaaudio/src/fifothe directory; - flowgraph, when using AAudio to play data, some simple processing will be done on the data, such as channel number conversion, etc. The flowgraph module is used to complete these processes, and the relevant code is located in the directory
libaaudio/src/flowgraph; - client, an audio stream implemented based on modules such as binding, fifo and flowgraph. The relevant code is located in
libaaudio/src/clientthe directory; - utility, some utility programs, related codes are located in
libaaudio/src/utilitythe directory.
From a module perspective, the implementation structure of the AAudio library is as shown below:
The analysis of Android AAudio implementation here focuses on the audio stream based on mmap shared memory to transfer audio data.
Open audio stream
Users of the AAudio interface open an audio stream AAudioStreamBuilder_openStream()based on the audio stream configuration in the interface . AAudioStreamBuilderThe definition of this function (located in frameworks/av/media/libaaudio/src/flowgraph/AudioProcessorBase.h ) is as follows:
AAUDIO_API aaudio_result_t AAudioStreamBuilder_openStream(AAudioStreamBuilder* builder,
AAudioStream** streamPtr)
{
AudioStream *audioStream = nullptr;
aaudio_stream_id_t id = 0;
// Please leave these logs because they are very helpful when debugging.
ALOGI("%s() called ----------------------------------------", __func__);
AudioStreamBuilder *streamBuilder = COMMON_GET_FROM_BUILDER_OR_RETURN(streamPtr);
aaudio_result_t result = streamBuilder->build(&audioStream);
if (result == AAUDIO_OK) {
*streamPtr = (AAudioStream*) audioStream;
id = audioStream->getId();
} else {
*streamPtr = nullptr;
}
ALOGI("%s() returns %d = %s for s#%u ----------------",
__func__, result, AAudio_convertResultToText(result), id);
return result;
}
AAudio actually AudioStreamBuilder::build()opens the audio stream, which creates and initializes the audio stream object AudioStreamand returns it to the caller. By the way, the and seen in the AAudio interface header file aaudio/AAudio.h are not real structures. These two structures are probably no different from and the actual structures are and .struct AAudioStreamStructstruct AAudioStreamBuilderStructvoidaaudio::AudioStreamaaudio::AudioStreamBuilder
AudioStreamBuilder::build(AudioStream** streamPtr)The definition of the function (located in frameworks/av/media/libaaudio/src/core/AudioStreamBuilder.cpp ) is as follows:
static aaudio_result_t builder_createStream(aaudio_direction_t direction,
aaudio_sharing_mode_t sharingMode,
bool tryMMap,
android::sp<AudioStream> &stream) {
aaudio_result_t result = AAUDIO_OK;
switch (direction) {
case AAUDIO_DIRECTION_INPUT:
if (tryMMap) {
stream = new AudioStreamInternalCapture(AAudioBinderClient::getInstance(),
false);
} else {
stream = new AudioStreamRecord();
}
break;
case AAUDIO_DIRECTION_OUTPUT:
if (tryMMap) {
stream = new AudioStreamInternalPlay(AAudioBinderClient::getInstance(),
false);
} else {
stream = new AudioStreamTrack();
}
break;
default:
ALOGE("%s() bad direction = %d", __func__, direction);
result = AAUDIO_ERROR_ILLEGAL_ARGUMENT;
}
return result;
}
// Try to open using MMAP path if that is allowed.
// Fall back to Legacy path if MMAP not available.
// Exact behavior is controlled by MMapPolicy.
aaudio_result_t AudioStreamBuilder::build(AudioStream** streamPtr) {
if (streamPtr == nullptr) {
ALOGE("%s() streamPtr is null", __func__);
return AAUDIO_ERROR_NULL;
}
*streamPtr = nullptr;
logParameters();
aaudio_result_t result = validate();
if (result != AAUDIO_OK) {
return result;
}
// The API setting is the highest priority.
aaudio_policy_t mmapPolicy = AudioGlobal_getMMapPolicy();
// If not specified then get from a system property.
if (mmapPolicy == AAUDIO_UNSPECIFIED) {
mmapPolicy = AAudioProperty_getMMapPolicy();
}
// If still not specified then use the default.
if (mmapPolicy == AAUDIO_UNSPECIFIED) {
mmapPolicy = AAUDIO_MMAP_POLICY_DEFAULT;
}
int32_t mapExclusivePolicy = AAudioProperty_getMMapExclusivePolicy();
if (mapExclusivePolicy == AAUDIO_UNSPECIFIED) {
mapExclusivePolicy = AAUDIO_MMAP_EXCLUSIVE_POLICY_DEFAULT;
}
aaudio_sharing_mode_t sharingMode = getSharingMode();
if ((sharingMode == AAUDIO_SHARING_MODE_EXCLUSIVE)
&& (mapExclusivePolicy == AAUDIO_POLICY_NEVER)) {
ALOGD("%s() EXCLUSIVE sharing mode not supported. Use SHARED.", __func__);
sharingMode = AAUDIO_SHARING_MODE_SHARED;
setSharingMode(sharingMode);
}
bool allowMMap = mmapPolicy != AAUDIO_POLICY_NEVER;
bool allowLegacy = mmapPolicy != AAUDIO_POLICY_ALWAYS;
// TODO Support other performance settings in MMAP mode.
// Disable MMAP if low latency not requested.
if (getPerformanceMode() != AAUDIO_PERFORMANCE_MODE_LOW_LATENCY) {
ALOGD("%s() MMAP not used because AAUDIO_PERFORMANCE_MODE_LOW_LATENCY not requested.",
__func__);
allowMMap = false;
}
// SessionID and Effects are only supported in Legacy mode.
if (getSessionId() != AAUDIO_SESSION_ID_NONE) {
ALOGD("%s() MMAP not used because sessionId specified.", __func__);
allowMMap = false;
}
if (!allowMMap && !allowLegacy) {
ALOGE("%s() no backend available: neither MMAP nor legacy path are allowed", __func__);
return AAUDIO_ERROR_ILLEGAL_ARGUMENT;
}
setPrivacySensitive(false);
if (mPrivacySensitiveReq == PRIVACY_SENSITIVE_DEFAULT) {
// When not explicitly requested, set privacy sensitive mode according to input preset:
// communication and camcorder captures are considered privacy sensitive by default.
aaudio_input_preset_t preset = getInputPreset();
if (preset == AAUDIO_INPUT_PRESET_CAMCORDER
|| preset == AAUDIO_INPUT_PRESET_VOICE_COMMUNICATION) {
setPrivacySensitive(true);
}
} else if (mPrivacySensitiveReq == PRIVACY_SENSITIVE_ENABLED) {
setPrivacySensitive(true);
}
android::sp<AudioStream> audioStream;
result = builder_createStream(getDirection(), sharingMode, allowMMap, audioStream);
if (result == AAUDIO_OK) {
// Open the stream using the parameters from the builder.
result = audioStream->open(*this);
if (result != AAUDIO_OK) {
bool isMMap = audioStream->isMMap();
if (isMMap && allowLegacy) {
ALOGV("%s() MMAP stream did not open so try Legacy path", __func__);
// If MMAP stream failed to open then TRY using a legacy stream.
result = builder_createStream(getDirection(), sharingMode,
false, audioStream);
if (result == AAUDIO_OK) {
result = audioStream->open(*this);
}
}
}
if (result == AAUDIO_OK) {
audioStream->registerPlayerBase();
audioStream->logOpenActual();
*streamPtr = startUsingStream(audioStream);
} // else audioStream will go out of scope and be deleted
}
return result;
}
The MMap policy is one of the most important parameters when creating and opening an audio stream. AudioStreamBuilder::build()The main process of opening an audio stream is as follows:
- To obtain the MMap policy, first obtain it from the API settings. If it is not set, obtain it from the system properties;
- Get the MMap exclusive policy from system properties;
- Correct the sharing mode according to the MMap exclusive policy and the set sharing mode;
- Check the performance mode and session ID. SessionID and sound effects are only supported in traditional mode. If the performance mode is not low-latency mode, use traditional mode;
- Create an audio stream through
builder_createStream()the function, which will create different types of audio stream objects according to the MMap mode and input and output types; - Execute
open()the operation of the audio stream object and open the audio stream; - If a request to open an MMap-mode audio stream fails, an attempt is made to create and open a legacy-mode audio stream.
The interface implementation of AAudio is relatively robust. When the low-latency mode cannot work properly due to various reasons, it will fall back to the traditional mode.
The hierarchical structure of audio stream objects in AAudio is as follows:
It is not difficult to see that AudioStreamInternalPlayand AudioStreamInternalCaptureare respectively the playback audio stream and collection audio stream types in MMap mode, AudioStreamTrackand AudioStreamRecordare respectively the playback audio stream and collection audio stream types in traditional mode.
AAudio provides interfaces for setting and obtaining MMap policies. The definitions of these interfaces (located in frameworks/av/media/libaaudio/src/core/AudioGlobal.cpp ) are as follows:
static aaudio_policy_t g_MMapPolicy = AAUDIO_UNSPECIFIED;
aaudio_policy_t AudioGlobal_getMMapPolicy() {
return g_MMapPolicy;
}
aaudio_result_t AudioGlobal_setMMapPolicy(aaudio_policy_t policy) {
aaudio_result_t result = AAUDIO_OK;
switch(policy) {
case AAUDIO_UNSPECIFIED:
case AAUDIO_POLICY_NEVER:
case AAUDIO_POLICY_AUTO:
case AAUDIO_POLICY_ALWAYS:
g_MMapPolicy = policy;
break;
default:
result = AAUDIO_ERROR_ILLEGAL_ARGUMENT;
break;
}
return result;
}
The system property that defines the MMap policy is aaudio.mmap_policy, and the system property that defines the MMap exclusive policy is aaudio.mmap_exclusive_policy, you can see this in the frameworks/av/media/libaaudio/src/utility/AAudioUtilities.h file:
int32_t AAudioProperty_getMMapPolicy();
#define AAUDIO_PROP_MMAP_POLICY "aaudio.mmap_policy"
/**
* Read system property.
* @return AAUDIO_UNSPECIFIED, AAUDIO_POLICY_NEVER or AAUDIO_POLICY_AUTO or AAUDIO_POLICY_ALWAYS
*/
int32_t AAudioProperty_getMMapExclusivePolicy();
#define AAUDIO_PROP_MMAP_EXCLUSIVE_POLICY "aaudio.mmap_exclusive_policy"
The method to obtain system properties (located in frameworks/av/media/libaaudio/src/utility/AAudioUtilities.cpp ) is as follows:
static int32_t AAudioProperty_getMMapProperty(const char *propName,
int32_t defaultValue,
const char * caller) {
int32_t prop = property_get_int32(propName, defaultValue);
switch (prop) {
case AAUDIO_UNSPECIFIED:
case AAUDIO_POLICY_NEVER:
case AAUDIO_POLICY_ALWAYS:
case AAUDIO_POLICY_AUTO:
break;
default:
ALOGE("%s: invalid = %d", caller, prop);
prop = defaultValue;
break;
}
return prop;
}
int32_t AAudioProperty_getMMapPolicy() {
return AAudioProperty_getMMapProperty(AAUDIO_PROP_MMAP_POLICY,
AAUDIO_UNSPECIFIED, __func__);
}
int32_t AAudioProperty_getMMapExclusivePolicy() {
return AAudioProperty_getMMapProperty(AAUDIO_PROP_MMAP_EXCLUSIVE_POLICY,
AAUDIO_UNSPECIFIED, __func__);
}
This system property can be defined in the mk file. For example, in order to enable the AAOS version of the simulator to start the media.aaudioservice, you need to device/generic/car/emulator/audio/car_emulator_audio.mkadd the following lines to the file:
PRODUCT_PROPERTY_OVERRIDES += aaudio.mmap_policy=2
Although in the implementation of the AAudio library, the MMap policy set by the API has the highest priority, this setting in the client process cannot affect media.aaudiowhether the service is started, but aaudio.mmap_policythe setting of the system property can.
Objects of different audio stream types open()have different operations. Here's a look at AudioStreamInternalPlay::open()the definition of the function (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternalPlay.cpp ):
aaudio_result_t AudioStreamInternalPlay::open(const AudioStreamBuilder &builder) {
aaudio_result_t result = AudioStreamInternal::open(builder);
if (result == AAUDIO_OK) {
result = mFlowGraph.configure(getFormat(),
getSamplesPerFrame(),
getDeviceFormat(),
getDeviceChannelCount());
if (result != AAUDIO_OK) {
safeReleaseClose();
}
// Sample rate is constrained to common values by now and should not overflow.
int32_t numFrames = kRampMSec * getSampleRate() / AAUDIO_MILLIS_PER_SECOND;
mFlowGraph.setRampLengthInFrames(numFrames);
}
return result;
}
AudioStreamInternalPlay::open()Execute AudioStreamInternal::open(builder)and configure the data processing pipeline for processing audio playback data AAudioFlowGraph.
AudioStreamInternal::open(builder)The definition of the function (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternal.cpp ) is as follows:
aaudio_result_t AudioStreamInternal::open(const AudioStreamBuilder &builder) {
aaudio_result_t result = AAUDIO_OK;
int32_t framesPerBurst;
int32_t framesPerHardwareBurst;
AAudioStreamRequest request;
AAudioStreamConfiguration configurationOutput;
if (getState() != AAUDIO_STREAM_STATE_UNINITIALIZED) {
ALOGE("%s - already open! state = %d", __func__, getState());
return AAUDIO_ERROR_INVALID_STATE;
}
// Copy requested parameters to the stream.
result = AudioStream::open(builder);
if (result < 0) {
return result;
}
const int32_t burstMinMicros = AAudioProperty_getHardwareBurstMinMicros();
int32_t burstMicros = 0;
const audio_format_t requestedFormat = getFormat();
// We have to do volume scaling. So we prefer FLOAT format.
if (requestedFormat == AUDIO_FORMAT_DEFAULT) {
setFormat(AUDIO_FORMAT_PCM_FLOAT);
}
// Request FLOAT for the shared mixer or the device.
request.getConfiguration().setFormat(AUDIO_FORMAT_PCM_FLOAT);
// TODO b/182392769: use attribution source util
AttributionSourceState attributionSource;
attributionSource.uid = VALUE_OR_FATAL(android::legacy2aidl_uid_t_int32_t(getuid()));
attributionSource.pid = VALUE_OR_FATAL(android::legacy2aidl_pid_t_int32_t(getpid()));
attributionSource.packageName = builder.getOpPackageName();
attributionSource.attributionTag = builder.getAttributionTag();
attributionSource.token = sp<android::BBinder>::make();
// Build the request to send to the server.
request.setAttributionSource(attributionSource);
request.setSharingModeMatchRequired(isSharingModeMatchRequired());
request.setInService(isInService());
request.getConfiguration().setDeviceId(getDeviceId());
request.getConfiguration().setSampleRate(getSampleRate());
request.getConfiguration().setDirection(getDirection());
request.getConfiguration().setSharingMode(getSharingMode());
request.getConfiguration().setChannelMask(getChannelMask());
request.getConfiguration().setUsage(getUsage());
request.getConfiguration().setContentType(getContentType());
request.getConfiguration().setSpatializationBehavior(getSpatializationBehavior());
request.getConfiguration().setIsContentSpatialized(isContentSpatialized());
request.getConfiguration().setInputPreset(getInputPreset());
request.getConfiguration().setPrivacySensitive(isPrivacySensitive());
request.getConfiguration().setBufferCapacity(builder.getBufferCapacity());
mDeviceChannelCount = getSamplesPerFrame(); // Assume it will be the same. Update if not.
mServiceStreamHandle = mServiceInterface.openStream(request, configurationOutput);
if (mServiceStreamHandle < 0
&& (request.getConfiguration().getSamplesPerFrame() == 1
|| request.getConfiguration().getChannelMask() == AAUDIO_CHANNEL_MONO)
&& getDirection() == AAUDIO_DIRECTION_OUTPUT
&& !isInService()) {
// if that failed then try switching from mono to stereo if OUTPUT.
// Only do this in the client. Otherwise we end up with a mono mixer in the service
// that writes to a stereo MMAP stream.
ALOGD("%s() - openStream() returned %d, try switching from MONO to STEREO",
__func__, mServiceStreamHandle);
request.getConfiguration().setChannelMask(AAUDIO_CHANNEL_STEREO);
mServiceStreamHandle = mServiceInterface.openStream(request, configurationOutput);
}
if (mServiceStreamHandle < 0) {
return mServiceStreamHandle;
}
// This must match the key generated in oboeservice/AAudioServiceStreamBase.cpp
// so the client can have permission to log.
if (!mInService) {
// No need to log if it is from service side.
mMetricsId = std::string(AMEDIAMETRICS_KEY_PREFIX_AUDIO_STREAM)
+ std::to_string(mServiceStreamHandle);
}
android::mediametrics::LogItem(mMetricsId)
.set(AMEDIAMETRICS_PROP_PERFORMANCEMODE,
AudioGlobal_convertPerformanceModeToText(builder.getPerformanceMode()))
.set(AMEDIAMETRICS_PROP_SHARINGMODE,
AudioGlobal_convertSharingModeToText(builder.getSharingMode()))
.set(AMEDIAMETRICS_PROP_ENCODINGCLIENT,
android::toString(requestedFormat).c_str()).record();
result = configurationOutput.validate();
if (result != AAUDIO_OK) {
goto error;
}
// Save results of the open.
if (getChannelMask() == AAUDIO_UNSPECIFIED) {
setChannelMask(configurationOutput.getChannelMask());
}
mDeviceChannelCount = configurationOutput.getSamplesPerFrame();
setSampleRate(configurationOutput.getSampleRate());
setDeviceId(configurationOutput.getDeviceId());
setSessionId(configurationOutput.getSessionId());
setSharingMode(configurationOutput.getSharingMode());
setUsage(configurationOutput.getUsage());
setContentType(configurationOutput.getContentType());
setSpatializationBehavior(configurationOutput.getSpatializationBehavior());
setIsContentSpatialized(configurationOutput.isContentSpatialized());
setInputPreset(configurationOutput.getInputPreset());
// Save device format so we can do format conversion and volume scaling together.
setDeviceFormat(configurationOutput.getFormat());
result = mServiceInterface.getStreamDescription(mServiceStreamHandle, mEndPointParcelable);
if (result != AAUDIO_OK) {
goto error;
}
// Resolve parcelable into a descriptor.
result = mEndPointParcelable.resolve(&mEndpointDescriptor);
if (result != AAUDIO_OK) {
goto error;
}
// Configure endpoint based on descriptor.
mAudioEndpoint = std::make_unique<AudioEndpoint>();
result = mAudioEndpoint->configure(&mEndpointDescriptor, getDirection());
if (result != AAUDIO_OK) {
goto error;
}
framesPerHardwareBurst = mEndpointDescriptor.dataQueueDescriptor.framesPerBurst;
// Scale up the burst size to meet the minimum equivalent in microseconds.
// This is to avoid waking the CPU too often when the HW burst is very small
// or at high sample rates.
framesPerBurst = framesPerHardwareBurst;
do {
if (burstMicros > 0) { // skip first loop
framesPerBurst *= 2;
}
burstMicros = framesPerBurst * static_cast<int64_t>(1000000) / getSampleRate();
} while (burstMicros < burstMinMicros);
ALOGD("%s() original HW burst = %d, minMicros = %d => SW burst = %d\n",
__func__, framesPerHardwareBurst, burstMinMicros, framesPerBurst);
// Validate final burst size.
if (framesPerBurst < MIN_FRAMES_PER_BURST || framesPerBurst > MAX_FRAMES_PER_BURST) {
ALOGE("%s - framesPerBurst out of range = %d", __func__, framesPerBurst);
result = AAUDIO_ERROR_OUT_OF_RANGE;
goto error;
}
setFramesPerBurst(framesPerBurst); // only save good value
mBufferCapacityInFrames = mEndpointDescriptor.dataQueueDescriptor.capacityInFrames;
if (mBufferCapacityInFrames < getFramesPerBurst()
|| mBufferCapacityInFrames > MAX_BUFFER_CAPACITY_IN_FRAMES) {
ALOGE("%s - bufferCapacity out of range = %d", __func__, mBufferCapacityInFrames);
result = AAUDIO_ERROR_OUT_OF_RANGE;
goto error;
}
mClockModel.setSampleRate(getSampleRate());
mClockModel.setFramesPerBurst(framesPerHardwareBurst);
if (isDataCallbackSet()) {
mCallbackFrames = builder.getFramesPerDataCallback();
if (mCallbackFrames > getBufferCapacity() / 2) {
ALOGW("%s - framesPerCallback too big = %d, capacity = %d",
__func__, mCallbackFrames, getBufferCapacity());
result = AAUDIO_ERROR_OUT_OF_RANGE;
goto error;
} else if (mCallbackFrames < 0) {
ALOGW("%s - framesPerCallback negative", __func__);
result = AAUDIO_ERROR_OUT_OF_RANGE;
goto error;
}
if (mCallbackFrames == AAUDIO_UNSPECIFIED) {
mCallbackFrames = getFramesPerBurst();
}
const int32_t callbackBufferSize = mCallbackFrames * getBytesPerFrame();
mCallbackBuffer = std::make_unique<uint8_t[]>(callbackBufferSize);
}
// For debugging and analyzing the distribution of MMAP timestamps.
// For OUTPUT, use a NEGATIVE offset to move the CPU writes further BEFORE the HW reads.
// For INPUT, use a POSITIVE offset to move the CPU reads further AFTER the HW writes.
// You can use this offset to reduce glitching.
// You can also use this offset to force glitching. By iterating over multiple
// values you can reveal the distribution of the hardware timing jitter.
if (mAudioEndpoint->isFreeRunning()) { // MMAP?
int32_t offsetMicros = (getDirection() == AAUDIO_DIRECTION_OUTPUT)
? AAudioProperty_getOutputMMapOffsetMicros()
: AAudioProperty_getInputMMapOffsetMicros();
// This log is used to debug some tricky glitch issues. Please leave.
ALOGD_IF(offsetMicros, "%s() - %s mmap offset = %d micros",
__func__,
(getDirection() == AAUDIO_DIRECTION_OUTPUT) ? "output" : "input",
offsetMicros);
mTimeOffsetNanos = offsetMicros * AAUDIO_NANOS_PER_MICROSECOND;
}
setBufferSize(mBufferCapacityInFrames / 2); // Default buffer size to match Q
setState(AAUDIO_STREAM_STATE_OPEN);
return result;
error:
safeReleaseClose();
return result;
}
AudioStreamInternal::open(builder)The execution process of the function is roughly as follows:
- The execution
AudioStream::open(builder)function mainlyAudioStreamBuildercopies the audio stream configuration made by the user, such as sampling rate, channel number, data format, sharing mode and performance mode, etc. For some configuration items that the user has not set, appropriate default values will be set; - Construct the AAudio stream request object, that is,
AAudioStreamRequestobject; - Request the service to open the audio stream through the AAudio service interface
media.aaudio. The return value of the operation of opening the audio stream through the AAudio service interface includes two parts. One is the return value of the operation, which is themedia.aaudiohandle of the server-side audio stream opened for the service, and the other is used as an outgoing parameter. TheAAudioStreamConfigurationobject, which mainly contains the actual configuration information of the audio stream; - When requesting the service to open the stream through the AAudio service interface
media.aaudiofails, if the number of stream channels requested is mono, it will try to open it again with a two-channel stereo audio stream configuration; - Set the status of the current audio stream according to the actual audio stream configuration information returned when opening the audio stream;
- Obtain the audio stream description through the AAudio service interface, pass in the handle of the server audio stream, and obtain the
AudioEndpointParcelableaudio stream description of a type object. The shared memory-related information required for the AAudio client to communicate with the service is not passed through the objectmedia.aaudiowhen opening the stream.AAudioStreamConfigurationWhat is returned is whatAudioEndpointParcelableis returned here through the object; AudioEndpointParcelableThe communication-related shared memory information of the object is converted intoEndpointDescriptora description in the form;- Create and configure
AudioEndpointthe object, which is mainly used tomedia.aaudioexchange commands and audio data with the service; - According to the configuration of the system properties, that is,
aaudio.hw_burst_min_usecthe minimum duration of each data transmission, correct the configuration of the minimum data amount of each data transmission; - Check the audio frame capacity of the buffer;
- Set the clock model;
- Configuration data callback;
- Set the buffer size and set the status of the audio stream to open.
As can be seen from AudioStreamInternal::open(builder)the function, the audio stream implementation of mmap-based shared memory communication with devices and media.aaudioservices has a class-level structure as shown below:
Although the directory of code related to API interface implementation is named core, in the AAudio library, the audio stream is undoubtedly the center of the entire implementation.
binding
The binding module is mainly used to media.aaudiocommunicate with services. AAudio defines the AAudio service interface AAudioServiceInterface. AudioStreamInternal::open(builder)When opening the audio stream, request media.aaudiothe service to open the audio stream through this interface and obtain the stream description. The definition of this interface (located in frameworks/av/media/libaaudio/src/binding/AAudioServiceInterface.h ) is as follows:
class AAudioServiceInterface {
public:
AAudioServiceInterface() {};
virtual ~AAudioServiceInterface() = default;
virtual void registerClient(const android::sp<IAAudioClient>& client) = 0;
/**
* @param request info needed to create the stream
* @param configuration contains information about the created stream
* @return handle to the stream or a negative error
*/
virtual aaudio_handle_t openStream(const AAudioStreamRequest &request,
AAudioStreamConfiguration &configuration) = 0;
virtual aaudio_result_t closeStream(aaudio_handle_t streamHandle) = 0;
/* Get an immutable description of the in-memory queues
* used to communicate with the underlying HAL or Service.
*/
virtual aaudio_result_t getStreamDescription(aaudio_handle_t streamHandle,
AudioEndpointParcelable &parcelable) = 0;
/**
* Start the flow of data.
*/
virtual aaudio_result_t startStream(aaudio_handle_t streamHandle) = 0;
/**
* Stop the flow of data such that start() can resume without loss of data.
*/
virtual aaudio_result_t pauseStream(aaudio_handle_t streamHandle) = 0;
/**
* Stop the flow of data after data currently inthe buffer has played.
*/
virtual aaudio_result_t stopStream(aaudio_handle_t streamHandle) = 0;
/**
* Discard any data held by the underlying HAL or Service.
*/
virtual aaudio_result_t flushStream(aaudio_handle_t streamHandle) = 0;
/**
* Manage the specified thread as a low latency audio thread.
*/
virtual aaudio_result_t registerAudioThread(aaudio_handle_t streamHandle,
pid_t clientThreadId,
int64_t periodNanoseconds) = 0;
virtual aaudio_result_t unregisterAudioThread(aaudio_handle_t streamHandle,
pid_t clientThreadId) = 0;
virtual aaudio_result_t startClient(aaudio_handle_t streamHandle,
const android::AudioClient& client,
const audio_attributes_t *attr,
audio_port_handle_t *clientHandle) = 0;
virtual aaudio_result_t stopClient(aaudio_handle_t streamHandle,
audio_port_handle_t clientHandle) = 0;
};
} /* namespace aaudio */
AAudioBinderAdapterEncapsulate media.aaudiothe client proxy object of the service and implement AAudioServiceInterfacethe interface based on the client proxy object. AAudioBinderAdapterThe implementation (located in frameworks/av/media/libaaudio/src/binding/AAudioBinderAdapter.cpp ) is as follows:
namespace aaudio {
using android::aidl_utils::statusTFromBinderStatus;
using android::binder::Status;
AAudioBinderAdapter::AAudioBinderAdapter(IAAudioService* delegate)
: mDelegate(delegate) {}
void AAudioBinderAdapter::registerClient(const android::sp<IAAudioClient>& client) {
mDelegate->registerClient(client);
}
aaudio_handle_t AAudioBinderAdapter::openStream(const AAudioStreamRequest& request,
AAudioStreamConfiguration& config) {
aaudio_handle_t result;
StreamParameters params;
Status status = mDelegate->openStream(request.parcelable(),
¶ms,
&result);
if (!status.isOk()) {
result = AAudioConvert_androidToAAudioResult(statusTFromBinderStatus(status));
}
config = params;
return result;
}
aaudio_result_t AAudioBinderAdapter::closeStream(aaudio_handle_t streamHandle) {
aaudio_result_t result;
Status status = mDelegate->closeStream(streamHandle, &result);
if (!status.isOk()) {
result = AAudioConvert_androidToAAudioResult(statusTFromBinderStatus(status));
}
return result;
}
aaudio_result_t AAudioBinderAdapter::getStreamDescription(aaudio_handle_t streamHandle,
AudioEndpointParcelable& endpointOut) {
aaudio_result_t result;
Endpoint endpoint;
Status status = mDelegate->getStreamDescription(streamHandle,
&endpoint,
&result);
if (!status.isOk()) {
result = AAudioConvert_androidToAAudioResult(statusTFromBinderStatus(status));
}
endpointOut = std::move(endpoint);
return result;
}
aaudio_result_t AAudioBinderAdapter::startStream(aaudio_handle_t streamHandle) {
aaudio_result_t result;
Status status = mDelegate->startStream(streamHandle, &result);
if (!status.isOk()) {
result = AAudioConvert_androidToAAudioResult(statusTFromBinderStatus(status));
}
return result;
}
aaudio_result_t AAudioBinderAdapter::pauseStream(aaudio_handle_t streamHandle) {
aaudio_result_t result;
Status status = mDelegate->pauseStream(streamHandle, &result);
if (!status.isOk()) {
result = AAudioConvert_androidToAAudioResult(statusTFromBinderStatus(status));
}
return result;
}
aaudio_result_t AAudioBinderAdapter::stopStream(aaudio_handle_t streamHandle) {
aaudio_result_t result;
Status status = mDelegate->stopStream(streamHandle, &result);
if (!status.isOk()) {
result = AAudioConvert_androidToAAudioResult(statusTFromBinderStatus(status));
}
return result;
}
aaudio_result_t AAudioBinderAdapter::flushStream(aaudio_handle_t streamHandle) {
aaudio_result_t result;
Status status = mDelegate->flushStream(streamHandle, &result);
if (!status.isOk()) {
result = AAudioConvert_androidToAAudioResult(statusTFromBinderStatus(status));
}
return result;
}
aaudio_result_t AAudioBinderAdapter::registerAudioThread(aaudio_handle_t streamHandle,
pid_t clientThreadId,
int64_t periodNanoseconds) {
aaudio_result_t result;
Status status = mDelegate->registerAudioThread(streamHandle, clientThreadId, periodNanoseconds, &result);
if (!status.isOk()) {
result = AAudioConvert_androidToAAudioResult(statusTFromBinderStatus(status));
}
return result;
}
aaudio_result_t AAudioBinderAdapter::unregisterAudioThread(aaudio_handle_t streamHandle,
pid_t clientThreadId) {
aaudio_result_t result;
Status status = mDelegate->unregisterAudioThread(streamHandle, clientThreadId, &result);
if (!status.isOk()) {
result = AAudioConvert_androidToAAudioResult(statusTFromBinderStatus(status));
}
return result;
}
} // namespace aaudio
AAudioBinderAdapterBasically what it does is transfer the operation to media.aaudiothe client proxy object of the service and convert the return value appropriately.
AAudioClientThe binder interface is used media.aaudioby the service to notify the client of some events and AAudioBinderClient::AAudioClientimplement BnAAudioClientthe interface. AAudioBinderClient::AAudioClientAfter receiving the event notification, the event will be passed on AAudioBinderClient.
AAudioBinderClient::AdapterInheritance AAudioBinderAdapter, which AAudioBinderAdapteradds the logic of logging out death notification when destroyed.
AAudioBinderClientIt is the core of the binding module. It maintains the media.aaudioclient proxy object, object, etc. of the service AAudioClient. AAudioBinderAdapterThe client module accesses media.aaudiothe service through it.
AAudioBinderClientThe implementation of the function used to obtain AAudioServiceInterfacethe interface object getAAudioService()(located in frameworks/av/media/libaaudio/src/binding/AAudioBinderClient.cpp ) is as follows:
// TODO Share code with other service clients.
// Helper function to get access to the "AAudioService" service.
// This code was modeled after frameworks/av/media/libaudioclient/AudioSystem.cpp
std::shared_ptr<AAudioServiceInterface> AAudioBinderClient::getAAudioService() {
std::shared_ptr<AAudioServiceInterface> result;
sp<IAAudioService> aaudioService;
bool needToRegister = false;
{
Mutex::Autolock _l(mServiceLock);
if (mAdapter == nullptr) {
sp<IBinder> binder;
sp<IServiceManager> sm = defaultServiceManager();
// Try several times to get the service.
int retries = 4;
do {
binder = sm->getService(String16(AAUDIO_SERVICE_NAME)); // This will wait a while.
if (binder.get() != nullptr) {
break;
}
} while (retries-- > 0);
if (binder.get() != nullptr) {
// Ask for notification if the service dies.
status_t status = binder->linkToDeath(mAAudioClient);
// TODO review what we should do if this fails
if (status != NO_ERROR) {
ALOGE("%s() - linkToDeath() returned %d", __func__, status);
}
aaudioService = interface_cast<IAAudioService>(binder);
mAdapter.reset(new Adapter(aaudioService, mAAudioClient));
needToRegister = true;
// Make sure callbacks can be received by mAAudioClient
ProcessState::self()->startThreadPool();
} else {
ALOGE("AAudioBinderClient could not connect to %s", AAUDIO_SERVICE_NAME);
}
}
result = mAdapter;
}
// Do this outside the mutex lock.
if (needToRegister && aaudioService.get() != nullptr) { // new client?
aaudioService->registerClient(mAAudioClient);
}
return result;
}
AAudioBinderClient::getAAudioService()media.aaudioThe function obtains the client proxy object of the service from the service manager and creates AAudioServiceInterfacean interface object based on the object, and then registers the death notification and AAudioClient.
When AAudioBinderClientreceiving media.aaudioa service hangup notification, it will destroy the resources it maintains, such as media.aaudiothe client proxy object of the service, etc. This is done through dropAAudioService()the function (located in frameworks/av/media/libaaudio/src/binding/AAudioBinderClient.cpp ):
void AAudioBinderClient::dropAAudioService() {
Mutex::Autolock _l(mServiceLock);
mAdapter.reset();
}
Exchanging data with media.aaudioservices is the core part of the binding module. Specific data exchange mainly occurs in the following operations:
- When opening the audio stream, send
AAudioStreamRequesta request represented by an object, receiveAAudioStreamConfigurationa response representing the stream configuration information represented by an object, and the handle of the server-side audio stream; - Obtain the description of the audio stream, send the handle of the server audio stream, and receive
AudioEndpointParcelablethe audio stream description information represented by , mainly themedia.aaudioshared memory related information used for subsequent communication with the service; - Audio data reading and writing.
The data objects used to represent requests and responses when opening an audio stream have this structure:
android::content::AttributionSourceStateThe object mainly contains the information of the current process, such as package name, user ID, process ID, etc.; the AAudioStreamConfigurationobject mainly contains the configuration information of the audio stream, such as device ID, sampling rate, number of channels, content type, performance mode, etc. The data that can be passed across processes through Binder IPC is only Parcelableobjects that support mechanisms for serialization. AAudioStreamConfigurationThe data of the object StreamParametersis passed through the AIDL object, AAudioStreamConfigurationproviding parcelable()operations to create StreamParametersobjects based on the state of the current object, and supporting StreamParametersobject creation based on AAudioStreamConfigurationobjects. The object's data is passed AAudioStreamRequestthrough AIDL objects .StreamRequest
The data object related to the audio stream description has the following structure:
The objects in the yellow rectangular box in the figure are AIDL objects, which are directly used for Binder IPC data exchange.
The above figure SharedMemoryParcelableis used to represent a shared memory space bound to a certain file descriptor, such as an mmap on an audio device. It may be divided into multiple different areas. This memory can be shared using Binder or between processes. Sharing, the definition of this class (located in frameworks/av/media/libaaudio/src/binding/SharedMemoryParcelable.h ) is as follows:
class SharedMemoryParcelable {
public:
SharedMemoryParcelable() = default;
// Ctor from a parcelable representation.
// Since the parcelable object owns a unique FD, move semantics are provided to avoid the need
// to dupe.
explicit SharedMemoryParcelable(android::media::SharedFileRegion&& parcelable);
/**
* Make a dup() of the fd and store it for later use.
*
* @param fd
* @param sizeInBytes
*/
void setup(const android::base::unique_fd& fd, int32_t sizeInBytes);
// mmap() shared memory
aaudio_result_t resolve(int32_t offsetInBytes, int32_t sizeInBytes, void **regionAddressPtr);
// munmap() any mapped memory
aaudio_result_t close();
int32_t getSizeInBytes();
void dump();
// Extract a parcelable representation of this object.
// Since we own a unique FD, move semantics are provided to avoid the need to dupe.
android::media::SharedFileRegion parcelable() &&;
// Copy this instance. Duplicates the underlying FD.
SharedMemoryParcelable dup() const;
private:
#define MMAP_UNRESOLVED_ADDRESS reinterpret_cast<uint8_t*>(MAP_FAILED)
android::base::unique_fd mFd;
int64_t mSizeInBytes = 0;
int64_t mOffsetInBytes = 0;
uint8_t *mResolvedAddress = MMAP_UNRESOLVED_ADDRESS;
aaudio_result_t resolveSharedMemory(const android::base::unique_fd& fd);
aaudio_result_t validate() const;
};
} /* namespace aaudio */
SharedMemoryParcelableContains the shared memory and its associated file descriptor, the size of the memory block, the offset of the memory block, and the address of the memory block in the current process. Generally speaking, the file description is the index of the file descriptor table of the process. It is only valid for the current process that has the file open. It is a resource belonging to the process. When passing the file descriptor, it needs to be created in the file descriptor table of the target process. The corresponding item points to the original file, and Android's Binder mechanism provides the ability to pass file descriptors across processes. The file descriptor seen here can be understood as something where the actual value may be different from the original file descriptor in another process, but points to the same actual file.
SharedMemoryParcelableCreate from the AIDL object used directly for Binder IPC communication SharedFileRegion, or create the object based on the current object (located in frameworks/av/media/libaaudio/src/binding/SharedMemoryParcelable.cpp ) as follows:
SharedMemoryParcelable::SharedMemoryParcelable(SharedFileRegion&& parcelable) {
mFd = parcelable.fd.release();
mSizeInBytes = parcelable.size;
mOffsetInBytes = parcelable.offset;
}
SharedFileRegion SharedMemoryParcelable::parcelable() && {
SharedFileRegion result;
result.fd.reset(std::move(mFd));
result.size = mSizeInBytes;
result.offset = mOffsetInBytes;
return result;
}
SharedMemoryParcelable::resolve()The function is used to parse to obtain a certain shared memory area in the shared memory block. It will perform memory mapping when needed. The definition of this function (located in frameworks/av/media/libaaudio/src/binding/SharedMemoryParcelable.cpp ) is as follows :
aaudio_result_t SharedMemoryParcelable::resolveSharedMemory(const unique_fd& fd) {
mResolvedAddress = (uint8_t *) mmap(0, mSizeInBytes, PROT_READ | PROT_WRITE,
MAP_SHARED, fd.get(), 0);
if (mResolvedAddress == MMAP_UNRESOLVED_ADDRESS) {
ALOGE("mmap() failed for fd = %d, nBytes = %" PRId64 ", errno = %s",
fd.get(), mSizeInBytes, strerror(errno));
return AAUDIO_ERROR_INTERNAL;
}
return AAUDIO_OK;
}
aaudio_result_t SharedMemoryParcelable::resolve(int32_t offsetInBytes, int32_t sizeInBytes,
void **regionAddressPtr) {
if (offsetInBytes < 0) {
ALOGE("illegal offsetInBytes = %d", offsetInBytes);
return AAUDIO_ERROR_OUT_OF_RANGE;
} else if ((offsetInBytes + sizeInBytes) > mSizeInBytes) {
ALOGE("out of range, offsetInBytes = %d, "
"sizeInBytes = %d, mSizeInBytes = %" PRId64,
offsetInBytes, sizeInBytes, mSizeInBytes);
return AAUDIO_ERROR_OUT_OF_RANGE;
}
aaudio_result_t result = AAUDIO_OK;
if (mResolvedAddress == MMAP_UNRESOLVED_ADDRESS) {
if (mFd.get() != -1) {
result = resolveSharedMemory(mFd);
} else {
ALOGE("has no file descriptor for shared memory.");
result = AAUDIO_ERROR_INTERNAL;
}
}
if (result == AAUDIO_OK && mResolvedAddress != MMAP_UNRESOLVED_ADDRESS) {
*regionAddressPtr = mResolvedAddress + offsetInBytes;
ALOGV("mResolvedAddress = %p", mResolvedAddress);
ALOGV("offset by %d, *regionAddressPtr = %p", offsetInBytes, *regionAddressPtr);
}
return result;
}
SharedRegionParcelableDescribes SharedMemoryParcelablea shared memory region on the shared memory block described by. The definition of this class (located in frameworks/av/media/libaaudio/src/binding/SharedRegionParcelable.h ) is as follows:
namespace aaudio {
class SharedRegionParcelable {
public:
SharedRegionParcelable() = default;
// Construct based on a parcelable representation.
explicit SharedRegionParcelable(const SharedRegion& parcelable);
void setup(int32_t sharedMemoryIndex, int32_t offsetInBytes, int32_t sizeInBytes);
aaudio_result_t resolve(SharedMemoryParcelable *memoryParcels, void **regionAddressPtr);
bool isFileDescriptorSafe(SharedMemoryParcelable *memoryParcels);
void dump();
// Extract a parcelable representation of this object.
SharedRegion parcelable() const;
private:
int32_t mSharedMemoryIndex = -1;
int32_t mOffsetInBytes = 0;
int32_t mSizeInBytes = 0;
aaudio_result_t validate() const;
};
} /* namespace aaudio */
As can be seen from SharedMemoryParcelable::validate()the function, the shared memory block size range is (0 kB, 256 kB):
aaudio_result_t SharedMemoryParcelable::validate() const {
if (mSizeInBytes < 0 || mSizeInBytes >= MAX_MMAP_SIZE_BYTES) {
ALOGE("invalid mSizeInBytes = %" PRId64, mSizeInBytes);
return AAUDIO_ERROR_OUT_OF_RANGE;
}
if (mOffsetInBytes != 0) {
ALOGE("invalid mOffsetInBytes = %" PRId64, mOffsetInBytes);
return AAUDIO_ERROR_OUT_OF_RANGE;
}
return AAUDIO_OK;
}
SharedRegionParcelableContains SharedMemoryParcelableinformation pointing to a shared memory area in a shared memory block in an array. When executing, SharedRegionParcelable::resolve()obtain the address of this shared memory area in the current process:
aaudio_result_t SharedRegionParcelable::resolve(SharedMemoryParcelable *memoryParcels,
void **regionAddressPtr) {
if (mSizeInBytes == 0) {
*regionAddressPtr = nullptr;
return AAUDIO_OK;
}
if (mSharedMemoryIndex < 0) {
ALOGE("invalid mSharedMemoryIndex = %d", mSharedMemoryIndex);
return AAUDIO_ERROR_INTERNAL;
}
SharedMemoryParcelable *memoryParcel = &memoryParcels[mSharedMemoryIndex];
return memoryParcel->resolve(mOffsetInBytes, mSizeInBytes, regionAddressPtr);
}
The operating system provides the underlying shared memory infrastructure, but to truly communicate through shared memory between two processes, more protocol design is required, such as how to synchronize access to the shared memory area between the two processes. RingBufferParcelableIt encapsulates several shared memory areas and implements a data path for exchanging large blocks of data through shared memory. RingBufferParcelableThe definition of the class (located in frameworks/av/media/libaaudio/src/binding/RingBufferParcelable.h ) is as follows:
namespace aaudio {
class RingBufferParcelable {
public:
RingBufferParcelable() = default;
// Construct based on a parcelable representation.
explicit RingBufferParcelable(const RingBuffer& parcelable);
// TODO This assumes that all three use the same SharedMemoryParcelable
void setupMemory(int32_t sharedMemoryIndex,
int32_t dataMemoryOffset,
int32_t dataSizeInBytes,
int32_t readCounterOffset,
int32_t writeCounterOffset,
int32_t counterSizeBytes);
void setupMemory(int32_t sharedMemoryIndex,
int32_t dataMemoryOffset,
int32_t dataSizeInBytes);
int32_t getBytesPerFrame();
void setBytesPerFrame(int32_t bytesPerFrame);
int32_t getFramesPerBurst();
void setFramesPerBurst(int32_t framesPerBurst);
int32_t getCapacityInFrames();
void setCapacityInFrames(int32_t capacityInFrames);
bool isFileDescriptorSafe(SharedMemoryParcelable *memoryParcels);
aaudio_result_t resolve(SharedMemoryParcelable *memoryParcels, RingBufferDescriptor *descriptor);
void dump();
// Extract a parcelable representation of this object.
RingBuffer parcelable() const;
private:
SharedRegionParcelable mReadCounterParcelable;
SharedRegionParcelable mWriteCounterParcelable;
SharedRegionParcelable mDataParcelable;
int32_t mBytesPerFrame = 0; // index is in frames
int32_t mFramesPerBurst = 0; // for ISOCHRONOUS queues
int32_t mCapacityInFrames = 0; // zero if unused
RingbufferFlags mFlags = RingbufferFlags::NONE;
aaudio_result_t validate() const;
};
} /* namespace aaudio */
Specifically, RingBufferParcelablethree shared memory areas are encapsulated, which are used for read counters, write counters and audio data transfer respectively. The method of creating and creating AIDL objects RingBufferParcelablebased on AIDL objects (located in frameworks/av/media/libaaudio/src/binding/RingBufferParcelable.cpp ) is as follows:RingBufferRingBuffer
RingBufferParcelable::RingBufferParcelable(const RingBuffer& parcelable)
: mReadCounterParcelable(std::move(parcelable.readCounterParcelable)),
mWriteCounterParcelable(std::move(parcelable.writeCounterParcelable)),
mDataParcelable(std::move(parcelable.dataParcelable)),
mBytesPerFrame(parcelable.bytesPerFrame),
mFramesPerBurst(parcelable.framesPerBurst),
mCapacityInFrames(parcelable.capacityInFrames),
mFlags(static_cast<RingbufferFlags>(parcelable.flags)) {
static_assert(sizeof(mFlags) == sizeof(parcelable.flags));
}
RingBuffer RingBufferParcelable::parcelable() const {
RingBuffer result;
result.readCounterParcelable = std::move(mReadCounterParcelable).parcelable();
result.writeCounterParcelable = std::move(mWriteCounterParcelable).parcelable();
result.dataParcelable = std::move(mDataParcelable).parcelable();
result.bytesPerFrame = mBytesPerFrame;
result.framesPerBurst = mFramesPerBurst;
result.capacityInFrames = mCapacityInFrames;
static_assert(sizeof(mFlags) == sizeof(result.flags));
result.flags = static_cast<int32_t>(mFlags);
return result;
}
RingBufferParcelable::setupMemory()The function sets three memory areas on the same memory:
void RingBufferParcelable::setupMemory(int32_t sharedMemoryIndex,
int32_t dataMemoryOffset,
int32_t dataSizeInBytes,
int32_t readCounterOffset,
int32_t writeCounterOffset,
int32_t counterSizeBytes) {
mReadCounterParcelable.setup(sharedMemoryIndex, readCounterOffset, counterSizeBytes);
mWriteCounterParcelable.setup(sharedMemoryIndex, writeCounterOffset, counterSizeBytes);
mDataParcelable.setup(sharedMemoryIndex, dataMemoryOffset, dataSizeInBytes);
}
void RingBufferParcelable::setupMemory(int32_t sharedMemoryIndex,
int32_t dataMemoryOffset,
int32_t dataSizeInBytes) {
mReadCounterParcelable.setup(sharedMemoryIndex, 0, 0);
mWriteCounterParcelable.setup(sharedMemoryIndex, 0, 0);
mDataParcelable.setup(sharedMemoryIndex, dataMemoryOffset, dataSizeInBytes);
}
RingBufferParcelable::resolve()Function is used to obtain the addresses of three memory areas:
aaudio_result_t RingBufferParcelable::resolve(SharedMemoryParcelable *memoryParcels, RingBufferDescriptor *descriptor) {
aaudio_result_t result;
result = mReadCounterParcelable.resolve(memoryParcels,
(void **) &descriptor->readCounterAddress);
if (result != AAUDIO_OK) {
return result;
}
result = mWriteCounterParcelable.resolve(memoryParcels,
(void **) &descriptor->writeCounterAddress);
if (result != AAUDIO_OK) {
return result;
}
result = mDataParcelable.resolve(memoryParcels, (void **) &descriptor->dataAddress);
if (result != AAUDIO_OK) {
return result;
}
descriptor->bytesPerFrame = mBytesPerFrame;
descriptor->framesPerBurst = mFramesPerBurst;
descriptor->capacityInFrames = mCapacityInFrames;
descriptor->flags = mFlags;
return AAUDIO_OK;
}
An audio stream needs media.aaudioto maintain multiple data paths based on shared memory AudioEndpointParcelablewith the service, and maintain relevant information in an audio stream media.aaudiothat communicates with the service through shared memory, including SharedMemoryParcelableinformation on multiple shared memory blocks, and data paths for different purposes. RingBufferParcelableInformation. AudioEndpointParcelableThe definition of the class (located in frameworks/av/media/libaaudio/src/binding/AudioEndpointParcelable.h ) is as follows:
namespace aaudio {
/**
* Container for information about the message queues plus
* general stream information needed by AAudio clients.
* It contains no addresses, just sizes, offsets and file descriptors for
* shared memory that can be passed through Binder.
*/
class AudioEndpointParcelable {
public:
AudioEndpointParcelable() = default;
// Ctor/assignment from a parcelable representation.
// Since the parcelable object owns unique FDs (for shared memory blocks), move semantics are
// provided to avoid the need to dupe.
AudioEndpointParcelable(Endpoint&& parcelable);
AudioEndpointParcelable& operator=(Endpoint&& parcelable);
/**
* Add the file descriptor to the table.
* @return index in table or negative error
*/
int32_t addFileDescriptor(const android::base::unique_fd& fd, int32_t sizeInBytes);
aaudio_result_t resolve(EndpointDescriptor *descriptor);
aaudio_result_t close();
void dump();
// Extract a parcelable representation of this object.
// Since our shared memory objects own a unique FD, move semantics are provided to avoid the
// need to dupe.
Endpoint parcelable()&&;
public: // TODO add getters
// Set capacityInFrames to zero if Queue is unused.
RingBufferParcelable mUpMessageQueueParcelable; // server to client
RingBufferParcelable mDownMessageQueueParcelable; // to server
RingBufferParcelable mUpDataQueueParcelable; // eg. record, could share same queue
RingBufferParcelable mDownDataQueueParcelable; // eg. playback
private:
aaudio_result_t validate() const;
int32_t mNumSharedMemories = 0;
SharedMemoryParcelable mSharedMemories[MAX_SHARED_MEMORIES];
};
} /* namespace aaudio */
AudioEndpointParcelableThe object's information comes from the AIDL object Endpoint. AudioEndpointParcelable::resolve()The function parses the information of shared memory, establishes the memory mapping for shared memory in the current process, etc.:
aaudio_result_t AudioEndpointParcelable::resolve(EndpointDescriptor *descriptor) {
aaudio_result_t result = mUpMessageQueueParcelable.resolve(mSharedMemories,
&descriptor->upMessageQueueDescriptor);
if (result != AAUDIO_OK) return result;
result = mDownMessageQueueParcelable.resolve(mSharedMemories,
&descriptor->downMessageQueueDescriptor);
if (result != AAUDIO_OK) return result;
result = mDownDataQueueParcelable.resolve(mSharedMemories,
&descriptor->dataQueueDescriptor);
return result;
}
AudioEndpointParcelable, RingBufferParcelable, SharedRegionParcelableand SharedMemoryParcelableetc. are not used directly for media.aaudiocommunication with the service. AudioEndpointParcelable::resolve()Shared memory related information for function resolution is EndpointDescriptormaintained by . EndpointDescriptorThe definition (located in frameworks/av/media/libaaudio/src/binding/AAudioServiceDefinitions.h ) is as follows:
// This must be a fixed width so it can be in shared memory.
enum RingbufferFlags : uint32_t {
NONE = 0,
RATE_ISOCHRONOUS = 0x0001,
RATE_ASYNCHRONOUS = 0x0002,
COHERENCY_DMA = 0x0004,
COHERENCY_ACQUIRE_RELEASE = 0x0008,
COHERENCY_AUTO = 0x0010,
};
// This is not passed through Binder.
// Client side code will convert Binder data and fill this descriptor.
typedef struct RingBufferDescriptor_s {
uint8_t* dataAddress; // offset from read or write block
int64_t* writeCounterAddress;
int64_t* readCounterAddress;
int32_t bytesPerFrame; // index is in frames
int32_t framesPerBurst; // for ISOCHRONOUS queues
int32_t capacityInFrames; // zero if unused
RingbufferFlags flags;
} RingBufferDescriptor;
// This is not passed through Binder.
// Client side code will convert Binder data and fill this descriptor.
typedef struct EndpointDescriptor_s {
// Set capacityInFrames to zero if Queue is unused.
RingBufferDescriptor upMessageQueueDescriptor; // server to client
RingBufferDescriptor downMessageQueueDescriptor; // client to server
RingBufferDescriptor dataQueueDescriptor; // playback or capture
} EndpointDescriptor;
An audio stream and media.aaudioservice actually have three data channels, which are used for sending messages, receiving messages and audio data transmission.
fifo
It is not difficult to see that it is not very convenient to communicate directly EndpointDescriptorwith media.aaudiothe service. The fifo module encapsulates shared memory blocks used for inter-process communication and provides more convenient operations. The structural relationship of each class in the fifo module is as follows:
FifoBufferRepresents a data channel, FifoControllerBasewhich is used to maintain FifoBufferread counters and write counters. FifoBufferIndirectImplemented for shared memory blocks based on inter-process communication FifoBuffer, and FifoBufferAllocatedimplemented based on memory allocated on the heap FifoBuffer.
AAudio's audio stream AudioEndpointmaintains these data channels. AudioEndpoint::configure()Function (located in frameworks/av/media/libaaudio/src/client/AudioEndpoint.cpp ) EndpointDescriptorcreates FifoBufferan object based on:
// TODO Consider moving to a method in RingBufferDescriptor
static aaudio_result_t AudioEndpoint_validateQueueDescriptor(const char *type,
const RingBufferDescriptor *descriptor) {
if (descriptor == nullptr) {
ALOGE("AudioEndpoint_validateQueueDescriptor() NULL descriptor");
return AAUDIO_ERROR_NULL;
}
if (descriptor->capacityInFrames < 1
|| descriptor->capacityInFrames > RIDICULOUSLY_LARGE_BUFFER_CAPACITY) {
ALOGE("AudioEndpoint_validateQueueDescriptor() bad capacityInFrames = %d",
descriptor->capacityInFrames);
return AAUDIO_ERROR_OUT_OF_RANGE;
}
// Reject extreme values to catch bugs and prevent numeric overflows.
if (descriptor->bytesPerFrame < 1
|| descriptor->bytesPerFrame > RIDICULOUSLY_LARGE_FRAME_SIZE) {
ALOGE("AudioEndpoint_validateQueueDescriptor() bad bytesPerFrame = %d",
descriptor->bytesPerFrame);
return AAUDIO_ERROR_OUT_OF_RANGE;
}
if (descriptor->dataAddress == nullptr) {
ALOGE("AudioEndpoint_validateQueueDescriptor() NULL dataAddress");
return AAUDIO_ERROR_NULL;
}
ALOGV("AudioEndpoint_validateQueueDescriptor %s, dataAddress at %p ====================",
type,
descriptor->dataAddress);
ALOGV("AudioEndpoint_validateQueueDescriptor readCounter at %p, writeCounter at %p",
descriptor->readCounterAddress,
descriptor->writeCounterAddress);
// Try to READ from the data area.
// This code will crash if the mmap failed.
uint8_t value = descriptor->dataAddress[0];
ALOGV("AudioEndpoint_validateQueueDescriptor() dataAddress[0] = %d, then try to write",
(int) value);
// Try to WRITE to the data area.
descriptor->dataAddress[0] = value * 3;
ALOGV("AudioEndpoint_validateQueueDescriptor() wrote successfully");
if (descriptor->readCounterAddress) {
fifo_counter_t counter = *descriptor->readCounterAddress;
ALOGV("AudioEndpoint_validateQueueDescriptor() *readCounterAddress = %d, now write",
(int) counter);
*descriptor->readCounterAddress = counter;
ALOGV("AudioEndpoint_validateQueueDescriptor() wrote readCounterAddress successfully");
}
if (descriptor->writeCounterAddress) {
fifo_counter_t counter = *descriptor->writeCounterAddress;
ALOGV("AudioEndpoint_validateQueueDescriptor() *writeCounterAddress = %d, now write",
(int) counter);
*descriptor->writeCounterAddress = counter;
ALOGV("AudioEndpoint_validateQueueDescriptor() wrote writeCounterAddress successfully");
}
return AAUDIO_OK;
}
aaudio_result_t AudioEndpoint_validateDescriptor(const EndpointDescriptor *pEndpointDescriptor) {
aaudio_result_t result = AudioEndpoint_validateQueueDescriptor("messages",
&pEndpointDescriptor->upMessageQueueDescriptor);
if (result == AAUDIO_OK) {
result = AudioEndpoint_validateQueueDescriptor("data",
&pEndpointDescriptor->dataQueueDescriptor);
}
return result;
}
aaudio_result_t AudioEndpoint::configure(const EndpointDescriptor *pEndpointDescriptor,
aaudio_direction_t direction)
{
aaudio_result_t result = AudioEndpoint_validateDescriptor(pEndpointDescriptor);
if (result != AAUDIO_OK) {
return result;
}
// ============================ up message queue =============================
const RingBufferDescriptor *descriptor = &pEndpointDescriptor->upMessageQueueDescriptor;
if(descriptor->bytesPerFrame != sizeof(AAudioServiceMessage)) {
ALOGE("configure() bytesPerFrame != sizeof(AAudioServiceMessage) = %d",
descriptor->bytesPerFrame);
return AAUDIO_ERROR_INTERNAL;
}
if(descriptor->readCounterAddress == nullptr || descriptor->writeCounterAddress == nullptr) {
ALOGE("configure() NULL counter address");
return AAUDIO_ERROR_NULL;
}
// Prevent memory leak and reuse.
if(mUpCommandQueue != nullptr || mDataQueue != nullptr) {
ALOGE("configure() endpoint already used");
return AAUDIO_ERROR_INTERNAL;
}
mUpCommandQueue = std::make_unique<FifoBufferIndirect>(
descriptor->bytesPerFrame,
descriptor->capacityInFrames,
descriptor->readCounterAddress,
descriptor->writeCounterAddress,
descriptor->dataAddress
);
// ============================ data queue =============================
descriptor = &pEndpointDescriptor->dataQueueDescriptor;
ALOGV("configure() data framesPerBurst = %d", descriptor->framesPerBurst);
ALOGV("configure() data readCounterAddress = %p",
descriptor->readCounterAddress);
// An example of free running is when the other side is read or written by hardware DMA
// or a DSP. It does not update its counter so we have to update it.
int64_t *remoteCounter = (direction == AAUDIO_DIRECTION_OUTPUT)
? descriptor->readCounterAddress // read by other side
: descriptor->writeCounterAddress; // written by other side
mFreeRunning = (remoteCounter == nullptr);
ALOGV("configure() mFreeRunning = %d", mFreeRunning ? 1 : 0);
int64_t *readCounterAddress = (descriptor->readCounterAddress == nullptr)
? &mDataReadCounter
: descriptor->readCounterAddress;
int64_t *writeCounterAddress = (descriptor->writeCounterAddress == nullptr)
? &mDataWriteCounter
: descriptor->writeCounterAddress;
// Clear buffer to avoid an initial glitch on some devices.
size_t bufferSizeBytes = descriptor->capacityInFrames * descriptor->bytesPerFrame;
memset(descriptor->dataAddress, 0, bufferSizeBytes);
mDataQueue = std::make_unique<FifoBufferIndirect>(
descriptor->bytesPerFrame,
descriptor->capacityInFrames,
readCounterAddress,
writeCounterAddress,
descriptor->dataAddress
);
uint32_t threshold = descriptor->capacityInFrames / 2;
mDataQueue->setThreshold(threshold);
return result;
}
Each audio stream actually uses two data channels, one is used for media.aaudiothe service to send messages to the AAudio client; the other is used for audio data transmission.
AudioEndpoint::configure()The execution process is roughly as follows:
- Verify the object passed in
EndpointDescriptor; - Create a data channel for
media.aaudiothe service to send messages to the AAudio clientFifoBufferIndirect. The format of the message will beAAudioServiceMessage; - Created for the data channel used for audio data transfer
FifoBufferIndirect.
FifoBufferMainly provides read and write operations on the data channel. The general process of read and write operations is to first FifoBufferobtain the package of the available memory area from , and then read and write the available memory area. The wrapping of the available memory area is struct WrappingBufferdescribed by , and the definition of this structure (located in frameworks/av/media/libaaudio/src/fifo/FifoBuffer.h ) is as follows:
struct WrappingBuffer {
enum {
SIZE = 2
};
void *data[SIZE];
int32_t numFrames[SIZE];
};
Since the memory block of the data channel is a ring buffer, the available memory area may span the buffer boundary, so the memory areas at the buffer tail and buffer head are described separately. getFullDataAvailable(WrappingBuffer *wrappingBuffer)Used to obtain a readable data area and getEmptyRoomAvailable(WrappingBuffer *wrappingBuffer)a writable data area. The definitions of these two functions (located in frameworks/av/media/libaaudio/src/fifo/FifoBuffer.cpp ) are as follows:
void FifoBuffer::fillWrappingBuffer(WrappingBuffer *wrappingBuffer,
int32_t framesAvailable,
int32_t startIndex) {
wrappingBuffer->data[1] = nullptr;
wrappingBuffer->numFrames[1] = 0;
uint8_t *storage = getStorage();
if (framesAvailable > 0) {
fifo_frames_t capacity = mFifo->getCapacity();
uint8_t *source = &storage[convertFramesToBytes(startIndex)];
// Does the available data cross the end of the FIFO?
if ((startIndex + framesAvailable) > capacity) {
wrappingBuffer->data[0] = source;
fifo_frames_t firstFrames = capacity - startIndex;
wrappingBuffer->numFrames[0] = firstFrames;
wrappingBuffer->data[1] = &storage[0];
wrappingBuffer->numFrames[1] = framesAvailable - firstFrames;
} else {
wrappingBuffer->data[0] = source;
wrappingBuffer->numFrames[0] = framesAvailable;
}
} else {
wrappingBuffer->data[0] = nullptr;
wrappingBuffer->numFrames[0] = 0;
}
}
fifo_frames_t FifoBuffer::getFullDataAvailable(WrappingBuffer *wrappingBuffer) {
// The FIFO might be overfull so clip to capacity.
fifo_frames_t framesAvailable = std::min(mFifo->getFullFramesAvailable(),
mFifo->getCapacity());
fifo_frames_t startIndex = mFifo->getReadIndex();
fillWrappingBuffer(wrappingBuffer, framesAvailable, startIndex);
return framesAvailable;
}
fifo_frames_t FifoBuffer::getEmptyRoomAvailable(WrappingBuffer *wrappingBuffer) {
// The FIFO might have underrun so clip to capacity.
fifo_frames_t framesAvailable = std::min(mFifo->getEmptyFramesAvailable(),
mFifo->getCapacity());
fifo_frames_t startIndex = mFifo->getWriteIndex();
fillWrappingBuffer(wrappingBuffer, framesAvailable, startIndex);
return framesAvailable;
}
FifoBufferUsers of can obtain the available buffers through getFullDataAvailable(WrappingBuffer *wrappingBuffer)and , and then pass data or messages through the obtained buffers, or they can directly use the and operations. The definitions of these two functions (located in frameworks/av/media/libaaudio/src/ fifo/FifoBuffer.cpp ) as follows:getEmptyRoomAvailable(WrappingBuffer *wrappingBuffer)FifoBufferread()write()
fifo_frames_t FifoBuffer::read(void *buffer, fifo_frames_t numFrames) {
WrappingBuffer wrappingBuffer;
uint8_t *destination = (uint8_t *) buffer;
fifo_frames_t framesLeft = numFrames;
getFullDataAvailable(&wrappingBuffer);
// Read data in one or two parts.
int partIndex = 0;
while (framesLeft > 0 && partIndex < WrappingBuffer::SIZE) {
fifo_frames_t framesToRead = framesLeft;
fifo_frames_t framesAvailable = wrappingBuffer.numFrames[partIndex];
if (framesAvailable > 0) {
if (framesToRead > framesAvailable) {
framesToRead = framesAvailable;
}
int32_t numBytes = convertFramesToBytes(framesToRead);
memcpy(destination, wrappingBuffer.data[partIndex], numBytes);
destination += numBytes;
framesLeft -= framesToRead;
} else {
break;
}
partIndex++;
}
fifo_frames_t framesRead = numFrames - framesLeft;
mFifo->advanceReadIndex(framesRead);
return framesRead;
}
fifo_frames_t FifoBuffer::write(const void *buffer, fifo_frames_t numFrames) {
WrappingBuffer wrappingBuffer;
uint8_t *source = (uint8_t *) buffer;
fifo_frames_t framesLeft = numFrames;
getEmptyRoomAvailable(&wrappingBuffer);
// Read data in one or two parts.
int partIndex = 0;
while (framesLeft > 0 && partIndex < WrappingBuffer::SIZE) {
fifo_frames_t framesToWrite = framesLeft;
fifo_frames_t framesAvailable = wrappingBuffer.numFrames[partIndex];
if (framesAvailable > 0) {
if (framesToWrite > framesAvailable) {
framesToWrite = framesAvailable;
}
int32_t numBytes = convertFramesToBytes(framesToWrite);
memcpy(wrappingBuffer.data[partIndex], source, numBytes);
source += numBytes;
framesLeft -= framesToWrite;
} else {
break;
}
partIndex++;
}
fifo_frames_t framesWritten = numFrames - framesLeft;
mFifo->advanceWriteIndex(framesWritten);
return framesWritten;
}
FifoBufferThe general process of the read()and write()operations is as follows:
- Obtain the available buffer through
getFullDataAvailable(WrappingBuffer *wrappingBuffer)or ;getEmptyRoomAvailable(WrappingBuffer *wrappingBuffer) - Read some data;
- Update read counter or write counter.
Both the read counter and the write counter increase monotonically. The amount of readable data in the buffer is the write counter minus the read counter. The amount of writable data is the threshold minus the amount of readable data. The read pointer and the write pointer pass through the read counter and write The counter is obtained by dividing the remainder by the buffer capacity, as shown in the following code (located in frameworks/av/media/libaaudio/src/fifo/FifoControllerBase.cpp ):
FifoControllerBase::FifoControllerBase(fifo_frames_t capacity, fifo_frames_t threshold)
: mCapacity(capacity)
, mThreshold(threshold)
{
}
FifoControllerBase::~FifoControllerBase() {
}
fifo_frames_t FifoControllerBase::getFullFramesAvailable() {
fifo_frames_t temp = 0;
__builtin_sub_overflow(getWriteCounter(), getReadCounter(), &temp);
return temp;
}
fifo_frames_t FifoControllerBase::getReadIndex() {
// % works with non-power of two sizes
return (fifo_frames_t) ((uint64_t)getReadCounter() % mCapacity);
}
void FifoControllerBase::advanceReadIndex(fifo_frames_t numFrames) {
fifo_counter_t temp = 0;
__builtin_add_overflow(getReadCounter(), numFrames, &temp);
setReadCounter(temp);
}
fifo_frames_t FifoControllerBase::getEmptyFramesAvailable() {
return (int32_t)(mThreshold - getFullFramesAvailable());
}
fifo_frames_t FifoControllerBase::getWriteIndex() {
// % works with non-power of two sizes
return (fifo_frames_t) ((uint64_t)getWriteCounter() % mCapacity);
}
void FifoControllerBase::advanceWriteIndex(fifo_frames_t numFrames) {
fifo_counter_t temp = 0;
__builtin_add_overflow(getWriteCounter(), numFrames, &temp);
setWriteCounter(temp);
}
void FifoControllerBase::setThreshold(fifo_frames_t threshold) {
if (threshold > mCapacity) {
threshold = mCapacity;
} else if (threshold < 0) {
threshold = 0;
}
mThreshold = threshold;
}
FifoBufferUsers of cannot directly operate it FifoControllerBase, and FifoBufferread and write operations on are strictly controlled by it, so reading and writing will not cross the boundary.
Data processing in AAudio
The main goal of AAudio is high-performance audio collection and playback, but it can also do some simple processing of the data to be played. Specifically, it can receive audio sample data in the formats of float, , int16_t, int 24and , int32_tand convert it into audio sample data in these formats for delivery. Between receiving data and passing it out, it can perform volume ramping, audio sample range limiting, and mono to multi-channel processing on the data. The processing of data between receiving the data and passing the data out is only floatperformed for the data in the format. AudioSourceThe component used to receive the data converts the data in any format into floatthe data in the format. The component used to pass the data out. AudioSinkConvert floatdata in the format to the desired data format.
The structure of data processing related classes in AAudio is as follows:
AAudio is defined AudioProcessorBaseto represent audio data processing node components. The definition of this class (located in frameworks/av/media/libaaudio/src/flowgraph/AudioProcessorBase.h ) is as follows:
class AudioProcessorBase {
public:
virtual ~AudioProcessorBase() = default;
/**
* Perform custom function.
*
* @param framePosition index of first frame to be processed
* @param numFrames maximum number of frames requested for processing
* @return number of frames actually processed
*/
virtual int32_t onProcess(int64_t framePosition, int32_t numFrames) = 0;
/**
* If the framePosition is at or after the last frame position then call onProcess().
* This prevents infinite recursion in case of cyclic graphs.
* It also prevents nodes upstream from a branch from being executed twice.
*
* @param framePosition
* @param numFrames
* @return
*/
int32_t pullData(int64_t framePosition, int32_t numFrames);
protected:
int64_t mLastFramePosition = -1; // Start at -1 so that the first pull works.
private:
int32_t mFramesValid = 0; // num valid frames in the block
};
AudioProcessorBaseThe onProcess()function performs customized data processing operations pullData()to drive AudioProcessorBasedata processing. AudioProcessorBaseThe implementation is as follows:
int32_t AudioProcessorBase::pullData(int64_t framePosition, int32_t numFrames) {
if (framePosition > mLastFramePosition) {
mLastFramePosition = framePosition;
mFramesValid = onProcess(framePosition, numFrames);
}
return mFramesValid;
}
pullData()The method is called only when the incoming parameter framePositionis later than the last processed audio frame onProcess(), which prevents infinite recursion when the audio data processing graph forms a loop. It also prevents nodes upstream of the branch from being executed twice.
AAudio defines AudioPortconnectors to allow data to AudioProcessorBaseflow between different modules. The definitions of these classes (located in frameworks/av/media/libaaudio/src/flowgraph/AudioProcessorBase.h ) are as follows:
class AudioPort {
public:
AudioPort(AudioProcessorBase &parent, int32_t samplesPerFrame)
: mParent(parent)
, mSamplesPerFrame(samplesPerFrame) {
}
// Ports are often declared public. So let's make them non-copyable.
AudioPort(const AudioPort&) = delete;
AudioPort& operator=(const AudioPort&) = delete;
int32_t getSamplesPerFrame() const {
return mSamplesPerFrame;
}
protected:
AudioProcessorBase &mParent;
private:
const int32_t mSamplesPerFrame = 1;
};
/***************************************************************************/
/**
* This port contains a float type buffer.
* The size is framesPerBlock * samplesPerFrame).
*/
class AudioFloatBlockPort : public AudioPort {
public:
AudioFloatBlockPort(AudioProcessorBase &mParent,
int32_t samplesPerFrame,
int32_t framesPerBlock = kDefaultBlockSize
);
virtual ~AudioFloatBlockPort();
int32_t getFramesPerBlock() const {
return mFramesPerBlock;
}
protected:
/**
* @return buffer internal to the port or from a connected port
*/
virtual float *getBlock() {
return mSampleBlock;
}
private:
const int32_t mFramesPerBlock = 1;
float *mSampleBlock = nullptr; // allocated in constructor
};
/***************************************************************************/
/**
* The results of a module are stored in the buffer of the output ports.
*/
class AudioFloatOutputPort : public AudioFloatBlockPort {
public:
AudioFloatOutputPort(AudioProcessorBase &parent, int32_t samplesPerFrame)
: AudioFloatBlockPort(parent, samplesPerFrame) {
}
virtual ~AudioFloatOutputPort() = default;
using AudioFloatBlockPort::getBlock;
/**
* Call the parent module's onProcess() method.
* That may pull data from its inputs and recursively
* process the entire graph.
* @return number of frames actually pulled
*/
int32_t pullData(int64_t framePosition, int32_t numFrames);
/**
* Connect to the input of another module.
* An input port can only have one connection.
* An output port can have multiple connections.
* If you connect a second output port to an input port
* then it overwrites the previous connection.
*
* This not thread safe. Do not modify the graph topology form another thread while running.
*/
void connect(AudioFloatInputPort *port);
/**
* Disconnect from the input of another module.
* This not thread safe.
*/
void disconnect(AudioFloatInputPort *port);
};
/***************************************************************************/
class AudioFloatInputPort : public AudioFloatBlockPort {
public:
AudioFloatInputPort(AudioProcessorBase &parent, int32_t samplesPerFrame)
: AudioFloatBlockPort(parent, samplesPerFrame) {
}
virtual ~AudioFloatInputPort() = default;
/**
* If connected to an output port then this will return
* that output ports buffers.
* If not connected then it returns the input ports own buffer
* which can be loaded using setValue().
*/
float *getBlock() override;
/**
* Pull data from any output port that is connected.
*/
int32_t pullData(int64_t framePosition, int32_t numFrames);
/**
* Write every value of the float buffer.
* This value will be ignored if an output port is connected
* to this port.
*/
void setValue(float value) {
int numFloats = kDefaultBlockSize * getSamplesPerFrame();
float *buffer = getBlock();
for (int i = 0; i < numFloats; i++) {
*buffer++ = value;
}
}
/**
* Connect to the output of another module.
* An input port can only have one connection.
* An output port can have multiple connections.
* This not thread safe.
*/
void connect(AudioFloatOutputPort *port) {
assert(getSamplesPerFrame() == port->getSamplesPerFrame());
mConnected = port;
}
void disconnect(AudioFloatOutputPort *port) {
assert(mConnected == port);
(void) port;
mConnected = nullptr;
}
void disconnect() {
mConnected = nullptr;
}
private:
AudioFloatOutputPort *mConnected = nullptr;
};
AudioFloatBlockPortMainly maintains a buffer for data transfer. AudioProcessorBaseThe work of connecting different modules is mainly done by AudioFloatOutputPortand AudioFloatInputPort, which adds operations such as pullData(), connect()and . disconnect()The implementation of these AudioPort(located in frameworks/av/media/libaaudio/src/flowgraph/AudioProcessorBase.cpp ) is as follows:
/***************************************************************************/
AudioFloatBlockPort::AudioFloatBlockPort(AudioProcessorBase &parent,
int32_t samplesPerFrame,
int32_t framesPerBlock)
: AudioPort(parent, samplesPerFrame)
, mFramesPerBlock(framesPerBlock)
, mSampleBlock(NULL) {
int32_t numFloats = framesPerBlock * getSamplesPerFrame();
mSampleBlock = new float[numFloats]{0.0f};
}
AudioFloatBlockPort::~AudioFloatBlockPort() {
delete[] mSampleBlock;
}
/***************************************************************************/
int32_t AudioFloatOutputPort::pullData(int64_t framePosition, int32_t numFrames) {
numFrames = std::min(getFramesPerBlock(), numFrames);
return mParent.pullData(framePosition, numFrames);
}
// These need to be in the .cpp file because of forward cross references.
void AudioFloatOutputPort::connect(AudioFloatInputPort *port) {
port->connect(this);
}
void AudioFloatOutputPort::disconnect(AudioFloatInputPort *port) {
port->disconnect(this);
}
/***************************************************************************/
int32_t AudioFloatInputPort::pullData(int64_t framePosition, int32_t numFrames) {
return (mConnected == NULL)
? std::min(getFramesPerBlock(), numFrames)
: mConnected->pullData(framePosition, numFrames);
}
float *AudioFloatInputPort::getBlock() {
if (mConnected == NULL) {
return AudioFloatBlockPort::getBlock(); // loaded using setValue()
} else {
return mConnected->getBlock();
}
}
AudioPortThe way the design works is this:
AudioProcessorBaseThe module containsAudioFloatOutputPortobjects, and the interfaceAudioFloatOutputPortofpullData()is used to drive the audio data processing pipeline to process audio data, that is, to callAudioProcessorBaseandpullData()then performAudioProcessorBasethe data operation ofonProcess(). InAudioProcessorBasethe data operationonProcess()implementation of ,AudioProcessorBasethe data to be processed is obtained from elsewhere and put in after processingAudioFloatOutputPort. in the buffer;AudioProcessorBaseModules containAudioFloatInputPortobjects that are used toAudioProcessorBaseobtain data from other modules after they have been processed. It is not necessary forAudioFloatInputPortthe object to hold a reference to the module that contains it ;AudioProcessorBaseAudioFloatOutputPortThe and operationsAudioFloatInputPortofconnect()and are mainly used to connect thedisconnect()previousAudioProcessorBasemoduleAudioFloatInputPortwith the followingAudioProcessorBasemodule .AudioFloatOutputPort
The audio data processing pipeline composed of AudioProcessorBase, AudioFloatOutputPortand will be as shown below:AudioFloatOutputPort
AudioProcessorBaseData transfer between different modules is completed by AudioFloatOutputPortand . Data transfer between and in AudioFloatOutputPortthe same module is mainly completed by the data operation interface of .AudioProcessorBaseAudioFloatOutputPortAudioFloatOutputPortAudioProcessorBaseonProcess()
AAudio defines to represent the node component that AudioSourceprocesses input data, that is, converts a certain type of audio data into type data. The definition of this class (located in frameworks/av/media/libaaudio/src/flowgraph/AudioProcessorBase.h ) is as follows:float
class AudioSource : public AudioProcessorBase {
public:
explicit AudioSource(int32_t channelCount)
: output(*this, channelCount) {
}
virtual ~AudioSource() = default;
AudioFloatOutputPort output;
void setData(const void *data, int32_t numFrames) {
mData = data;
mSizeInFrames = numFrames;
mFrameIndex = 0;
}
protected:
const void *mData = nullptr;
int32_t mSizeInFrames = 0; // number of frames in mData
int32_t mFrameIndex = 0; // index of next frame to be processed
};
AudioSourceProvides an interface to the outside world to provide source data for the audio data processing pipeline, and passes the converted data to subsequent modules of the audio data processing pipeline setData()through it .AudioFloatOutputPortAudioProcessorBase
AAudio defines to represent the node component that AudioSinkprocesses output data, that is, converts type data into a certain type of audio data. The definition of this class (located in frameworks/av/media/libaaudio/src/flowgraph/AudioProcessorBase.h ) is as follows:float
class AudioSink : public AudioProcessorBase {
public:
explicit AudioSink(int32_t channelCount)
: input(*this, channelCount) {
}
virtual ~AudioSink() = default;
AudioFloatInputPort input;
/**
* Do nothing. The work happens in the read() method.
*
* @param framePosition index of first frame to be processed
* @param numFrames
* @return number of frames actually processed
*/
int32_t onProcess(int64_t framePosition, int32_t numFrames) override {
(void) framePosition;
(void) numFrames;
return 0;
};
virtual int32_t read(void *data, int32_t numFrames) = 0;
protected:
int32_t pull(int32_t numFrames);
private:
int64_t mFramePosition = 0;
};
AudioSinkThe onProcess()function is a no-op, it adds a new read()operation to complete the final data processing operation in the audio data processing pipeline and obtain the processed data. AudioSinkThe current design of will onProcess()be implemented as a no-op, which breaks the semantics of the inherited pullData()and onProcess()interfaces. For AudioSinkthe design of , there are other options:
(1). There is no need to add read()an operation;
(2). In addition to an AudioFloatInputPortobject, AudioSinkanother AudioFloatOutputPortobject is included;
(3). The entire audio data processing pipeline is driven by AudioFloatOutputPortthe object ; (4). The original interface completes The work is divided into two parts. The data processing part is placed in the interface. The processed data is saved in the buffer of . The data acquisition part is completed directly through the object.pullData()read()onProcess()AudioFloatOutputPortAudioFloatOutputPort
However, the current design can avoid a data copy. This is probably an example of efforts made for versatility or compromise for performance. Here are a few more examples of different types AudioProcessorBaseof modules. SourceI16It is used to int16_tconvert type source data into type floatdata AudioSource. The definition of this class (located in frameworks/av/media/libaaudio/src/flowgraph/SourceI16.h ) is as follows:
namespace flowgraph {
class SourceI16 : public AudioSource {
public:
explicit SourceI16(int32_t channelCount);
int32_t onProcess(int64_t framePosition, int32_t numFrames) override;
};
} /* namespace flowgraph */
The implementation of this class (located in frameworks/av/media/libaaudio/src/flowgraph/SourceI16.cpp ) is as follows:
using namespace flowgraph;
SourceI16::SourceI16(int32_t channelCount)
: AudioSource(channelCount) {
}
int32_t SourceI16::onProcess(int64_t framePosition, int32_t numFrames) {
float *floatData = output.getBlock();
int32_t channelCount = output.getSamplesPerFrame();
int32_t framesLeft = mSizeInFrames - mFrameIndex;
int32_t framesToProcess = std::min(numFrames, framesLeft);
int32_t numSamples = framesToProcess * channelCount;
const int16_t *shortBase = static_cast<const int16_t *>(mData);
const int16_t *shortData = &shortBase[mFrameIndex * channelCount];
#ifdef __ANDROID__
memcpy_to_float_from_i16(floatData, shortData, numSamples);
#else
for (int i = 0; i < numSamples; i++) {
*floatData++ = *shortData++ * (1.0f / 32768);
}
#endif
mFrameIndex += framesToProcess;
return framesToProcess;
}
MonoToMultiConverterLocated in the middle of the audio data processing pipeline, it is used to convert mono-channel data into multi-channel data. The definition of this class (located in frameworks/av/media/libaaudio/src/flowgraph/MonoToMultiConverter.h ) is as follows:
namespace flowgraph {
class MonoToMultiConverter : public AudioProcessorBase {
public:
explicit MonoToMultiConverter(int32_t channelCount);
virtual ~MonoToMultiConverter();
int32_t onProcess(int64_t framePosition, int32_t numFrames) override;
AudioFloatInputPort input;
AudioFloatOutputPort output;
};
} /* namespace flowgraph */
The implementation of this class (located in frameworks/av/media/libaaudio/src/flowgraph/MonoToMultiConverter.cpp ) is as follows:
using namespace flowgraph;
MonoToMultiConverter::MonoToMultiConverter(int32_t channelCount)
: input(*this, 1)
, output(*this, channelCount) {
}
MonoToMultiConverter::~MonoToMultiConverter() { }
int32_t MonoToMultiConverter::onProcess(int64_t framePosition, int32_t numFrames) {
int32_t framesToProcess = input.pullData(framePosition, numFrames);
const float *inputBuffer = input.getBlock();
float *outputBuffer = output.getBlock();
int32_t channelCount = output.getSamplesPerFrame();
// TODO maybe move to audio_util as audio_mono_to_multi()
for (int i = 0; i < framesToProcess; i++) {
// read one, write many
float sample = *inputBuffer++;
for (int channel = 0; channel < channelCount; channel++) {
*outputBuffer++ = sample;
}
}
return framesToProcess;
}
SinkI16It floatconverts type data into int16_ttype data AudioSink. The definition of this class (located in frameworks/av/media/libaaudio/src/flowgraph/SinkI16.h ) is as follows:
namespace flowgraph {
class SinkI16 : public AudioSink {
public:
explicit SinkI16(int32_t channelCount);
int32_t read(void *data, int32_t numFrames) override;
};
} /* namespace flowgraph */
The implementation of this class (located in frameworks/av/media/libaaudio/src/flowgraph/SinkI16.cpp ) is as follows:
using namespace flowgraph;
SinkI16::SinkI16(int32_t channelCount)
: AudioSink(channelCount) {}
int32_t SinkI16::read(void *data, int32_t numFrames) {
int16_t *shortData = (int16_t *) data;
const int32_t channelCount = input.getSamplesPerFrame();
int32_t framesLeft = numFrames;
while (framesLeft > 0) {
// Run the graph and pull data through the input port.
int32_t framesRead = pull(framesLeft);
if (framesRead <= 0) {
break;
}
const float *signal = input.getBlock();
int32_t numSamples = framesRead * channelCount;
#ifdef __ANDROID__
memcpy_to_i16_from_float(shortData, signal, numSamples);
shortData += numSamples;
signal += numSamples;
#else
for (int i = 0; i < numSamples; i++) {
int32_t n = (int32_t) (*signal++ * 32768.0f);
*shortData++ = std::min(INT16_MAX, std::max(INT16_MIN, n)); // clip
}
#endif
framesLeft -= framesRead;
}
return numFrames - framesLeft;
}
AAudio's audio data processing pipeline is AAudioFlowGraphmaintained by. The definition of this class (located in frameworks/av/media/libaaudio/src/client/AAudioFlowGraph.h ) is as follows:
class AAudioFlowGraph {
public:
/** Connect several modules together to convert from source to sink.
* This should only be called once for each instance.
*
* @param sourceFormat
* @param sourceChannelCount
* @param sinkFormat
* @param sinkChannelCount
* @return
*/
aaudio_result_t configure(audio_format_t sourceFormat,
int32_t sourceChannelCount,
audio_format_t sinkFormat,
int32_t sinkChannelCount);
void process(const void *source, void *destination, int32_t numFrames);
/**
* @param volume between 0.0 and 1.0
*/
void setTargetVolume(float volume);
void setRampLengthInFrames(int32_t numFrames);
private:
std::unique_ptr<flowgraph::AudioSource> mSource;
std::unique_ptr<flowgraph::RampLinear> mVolumeRamp;
std::unique_ptr<flowgraph::ClipToRange> mClipper;
std::unique_ptr<flowgraph::MonoToMultiConverter> mChannelConverter;
std::unique_ptr<flowgraph::AudioSink> mSink;
};
AAudioFlowGraphBuild an audio data processing pipeline during configuration and provide process()operations to process data through the audio data processing pipeline:
using namespace flowgraph;
aaudio_result_t AAudioFlowGraph::configure(audio_format_t sourceFormat,
int32_t sourceChannelCount,
audio_format_t sinkFormat,
int32_t sinkChannelCount) {
AudioFloatOutputPort *lastOutput = nullptr;
// TODO change back to ALOGD
ALOGI("%s() source format = 0x%08x, channels = %d, sink format = 0x%08x, channels = %d",
__func__, sourceFormat, sourceChannelCount, sinkFormat, sinkChannelCount);
switch (sourceFormat) {
case AUDIO_FORMAT_PCM_FLOAT:
mSource = std::make_unique<SourceFloat>(sourceChannelCount);
break;
case AUDIO_FORMAT_PCM_16_BIT:
mSource = std::make_unique<SourceI16>(sourceChannelCount);
break;
case AUDIO_FORMAT_PCM_24_BIT_PACKED:
mSource = std::make_unique<SourceI24>(sourceChannelCount);
break;
case AUDIO_FORMAT_PCM_32_BIT:
mSource = std::make_unique<SourceI32>(sourceChannelCount);
break;
default:
ALOGE("%s() Unsupported source format = %d", __func__, sourceFormat);
return AAUDIO_ERROR_UNIMPLEMENTED;
}
lastOutput = &mSource->output;
// Apply volume as a ramp to avoid pops.
mVolumeRamp = std::make_unique<RampLinear>(sourceChannelCount);
lastOutput->connect(&mVolumeRamp->input);
lastOutput = &mVolumeRamp->output;
// For a pure float graph, there is chance that the data range may be very large.
// So we should clip to a reasonable value that allows a little headroom.
if (sourceFormat == AUDIO_FORMAT_PCM_FLOAT && sinkFormat == AUDIO_FORMAT_PCM_FLOAT) {
mClipper = std::make_unique<ClipToRange>(sourceChannelCount);
lastOutput->connect(&mClipper->input);
lastOutput = &mClipper->output;
}
// Expand the number of channels if required.
if (sourceChannelCount == 1 && sinkChannelCount > 1) {
mChannelConverter = std::make_unique<MonoToMultiConverter>(sinkChannelCount);
lastOutput->connect(&mChannelConverter->input);
lastOutput = &mChannelConverter->output;
} else if (sourceChannelCount != sinkChannelCount) {
ALOGE("%s() Channel reduction not supported.", __func__);
return AAUDIO_ERROR_UNIMPLEMENTED;
}
switch (sinkFormat) {
case AUDIO_FORMAT_PCM_FLOAT:
mSink = std::make_unique<SinkFloat>(sinkChannelCount);
break;
case AUDIO_FORMAT_PCM_16_BIT:
mSink = std::make_unique<SinkI16>(sinkChannelCount);
break;
case AUDIO_FORMAT_PCM_24_BIT_PACKED:
mSink = std::make_unique<SinkI24>(sinkChannelCount);
break;
case AUDIO_FORMAT_PCM_32_BIT:
mSink = std::make_unique<SinkI32>(sinkChannelCount);
break;
default:
ALOGE("%s() Unsupported sink format = %d", __func__, sinkFormat);
return AAUDIO_ERROR_UNIMPLEMENTED;
}
lastOutput->connect(&mSink->input);
return AAUDIO_OK;
}
void AAudioFlowGraph::process(const void *source, void *destination, int32_t numFrames) {
mSource->setData(source, numFrames);
mSink->read(destination, numFrames);
}
/**
* @param volume between 0.0 and 1.0
*/
void AAudioFlowGraph::setTargetVolume(float volume) {
mVolumeRamp->setTarget(volume);
}
void AAudioFlowGraph::setRampLengthInFrames(int32_t numFrames) {
mVolumeRamp->setLengthInFrames(numFrames);
}
In this way, AAudio's audio data processing pipeline will eventually look like this:
Audio streaming implementation
Most of the resources required for the audio stream to run are allocated when the audio stream is opened. The audio stream will actually run when the application requests to start the audio stream. The interface implementation of AAudio request to start the audio stream (located in frameworks/av/media/libaaudio/src/core/AAudioAudio.cpp ) is as follows:
AAUDIO_API aaudio_result_t AAudioStream_requestStart(AAudioStream* stream)
{
AudioStream *audioStream = convertAAudioStreamToAudioStream(stream);
aaudio_stream_id_t id = audioStream->getId();
ALOGD("%s(s#%u) called --------------", __func__, id);
aaudio_result_t result = audioStream->systemStart();
ALOGD("%s(s#%u) returned %d ---------", __func__, id, result);
return result;
}
This function calls AudioStream::systemStart()the function, which is defined (located in frameworks/av/media/libaaudio/src/core/AudioStream.cpp ) as follows:
aaudio_result_t AudioStream::systemStart() {
if (collidesWithCallback()) {
ALOGE("%s cannot be called from a callback!", __func__);
return AAUDIO_ERROR_INVALID_STATE;
}
std::lock_guard<std::mutex> lock(mStreamLock);
switch (getState()) {
// Is this a good time to start?
case AAUDIO_STREAM_STATE_OPEN:
case AAUDIO_STREAM_STATE_PAUSING:
case AAUDIO_STREAM_STATE_PAUSED:
case AAUDIO_STREAM_STATE_STOPPING:
case AAUDIO_STREAM_STATE_STOPPED:
case AAUDIO_STREAM_STATE_FLUSHING:
case AAUDIO_STREAM_STATE_FLUSHED:
break; // Proceed with starting.
// Already started?
case AAUDIO_STREAM_STATE_STARTING:
case AAUDIO_STREAM_STATE_STARTED:
ALOGW("%s() stream was already started, state = %s", __func__,
AudioGlobal_convertStreamStateToText(getState()));
return AAUDIO_ERROR_INVALID_STATE;
// Don't start when the stream is dead!
case AAUDIO_STREAM_STATE_DISCONNECTED:
case AAUDIO_STREAM_STATE_CLOSING:
case AAUDIO_STREAM_STATE_CLOSED:
default:
ALOGW("%s() stream is dead, state = %s", __func__,
AudioGlobal_convertStreamStateToText(getState()));
return AAUDIO_ERROR_INVALID_STATE;
}
aaudio_result_t result = requestStart_l();
if (result == AAUDIO_OK) {
// We only call this for logging in "dumpsys audio". So ignore return code.
(void) mPlayerBase->startWithStatus(getDeviceId());
}
return result;
}
AudioStream::systemStart()The function checks the current status of the audio stream; when the status allows startup, it requests the specific audio stream to perform the startup action, that is requestStart_l(). AudioStreamInternal::requestStart_l()The definition of the function (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternal.cpp ) is as follows:
aaudio_result_t AudioStreamInternal::requestStart_l()
{
int64_t startTime;
if (mServiceStreamHandle == AAUDIO_HANDLE_INVALID) {
ALOGD("requestStart() mServiceStreamHandle invalid");
return AAUDIO_ERROR_INVALID_STATE;
}
if (isActive()) {
ALOGD("requestStart() already active");
return AAUDIO_ERROR_INVALID_STATE;
}
aaudio_stream_state_t originalState = getState();
if (originalState == AAUDIO_STREAM_STATE_DISCONNECTED) {
ALOGD("requestStart() but DISCONNECTED");
return AAUDIO_ERROR_DISCONNECTED;
}
setState(AAUDIO_STREAM_STATE_STARTING);
// Clear any stale timestamps from the previous run.
drainTimestampsFromService();
prepareBuffersForStart(); // tell subclasses to get ready
aaudio_result_t result = mServiceInterface.startStream(mServiceStreamHandle);
if (result == AAUDIO_ERROR_INVALID_HANDLE) {
ALOGD("%s() INVALID_HANDLE, stream was probably stolen", __func__);
// Stealing was added in R. Coerce result to improve backward compatibility.
result = AAUDIO_ERROR_DISCONNECTED;
setState(AAUDIO_STREAM_STATE_DISCONNECTED);
}
startTime = AudioClock::getNanoseconds();
mClockModel.start(startTime);
mNeedCatchUp.request(); // Ask data processing code to catch up when first timestamp received.
// Start data callback thread.
if (result == AAUDIO_OK && isDataCallbackSet()) {
// Launch the callback loop thread.
int64_t periodNanos = mCallbackFrames
* AAUDIO_NANOS_PER_SECOND
/ getSampleRate();
mCallbackEnabled.store(true);
result = createThread_l(periodNanos, aaudio_callback_thread_proc, this);
}
if (result != AAUDIO_OK) {
setState(originalState);
}
return result;
}
. . . . . .
aaudio_result_t AudioStreamInternal::drainTimestampsFromService() {
aaudio_result_t result = AAUDIO_OK;
while (result == AAUDIO_OK) {
AAudioServiceMessage message;
if (!mAudioEndpoint) {
break;
}
if (mAudioEndpoint->readUpCommand(&message) != 1) {
break; // no command this time, no problem
}
switch (message.what) {
// ignore most messages
case AAudioServiceMessage::code::TIMESTAMP_SERVICE:
case AAudioServiceMessage::code::TIMESTAMP_HARDWARE:
break;
case AAudioServiceMessage::code::EVENT:
result = onEventFromServer(&message);
break;
default:
ALOGE("%s - unrecognized message.what = %d", __func__, (int) message.what);
result = AAUDIO_ERROR_INTERNAL;
break;
}
}
return result;
}
The execution process of this function is as follows:
- Check the current state of the audio stream and set the current state to
AAUDIO_STREAM_STATE_STARTING; - Clear the messages sent by the server;
- Requests the audio stream subclass to prepare for startup and prepare the buffer;
- Request the service to start the audio stream through the AAudio service interface
media.aaudio; - Set the current time as the start time of the synchronized clock model to facilitate prediction of the audio stream position at a given time point;
- There are two types of data transmission for audio streams: push mode and pull mode. One is that the application synchronously takes out the audio data and sends it into the audio stream through
read()the / operation provided by the audio stream; the other is to set a data callback for the audio stream. Thiswrite()When AAudio starts a thread for the audio stream, it first completes data transfer with the application through callbacks, and then writes or reads the data into the data stream. When a data callback is set, the callback execution cycle is calculated and a thread is requested to be started.
The tasks performed by the thread started for the audio stream (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternal.cpp ) are as follows:
static void *aaudio_callback_thread_proc(void *context)
{
AudioStreamInternal *stream = (AudioStreamInternal *)context;
//LOGD("oboe_callback_thread, stream = %p", stream);
if (stream != NULL) {
return stream->callbackLoop();
} else {
return NULL;
}
}
The callback loop for playing the audio stream (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternalPlay.cpp ) is as follows:
void *AudioStreamInternalPlay::callbackLoop() {
ALOGD("%s() entering >>>>>>>>>>>>>>>", __func__);
aaudio_result_t result = AAUDIO_OK;
aaudio_data_callback_result_t callbackResult = AAUDIO_CALLBACK_RESULT_CONTINUE;
if (!isDataCallbackSet()) return NULL;
int64_t timeoutNanos = calculateReasonableTimeout(mCallbackFrames);
// result might be a frame count
while (mCallbackEnabled.load() && isActive() && (result >= 0)) {
// Call application using the AAudio callback interface.
callbackResult = maybeCallDataCallback(mCallbackBuffer.get(), mCallbackFrames);
if (callbackResult == AAUDIO_CALLBACK_RESULT_CONTINUE) {
// Write audio data to stream. This is a BLOCKING WRITE!
result = write(mCallbackBuffer.get(), mCallbackFrames, timeoutNanos);
if ((result != mCallbackFrames)) {
if (result >= 0) {
// Only wrote some of the frames requested. Must have timed out.
result = AAUDIO_ERROR_TIMEOUT;
}
maybeCallErrorCallback(result);
break;
}
} else if (callbackResult == AAUDIO_CALLBACK_RESULT_STOP) {
ALOGD("%s(): callback returned AAUDIO_CALLBACK_RESULT_STOP", __func__);
result = systemStopInternal();
break;
}
}
ALOGD("%s() exiting, result = %d, isActive() = %d <<<<<<<<<<<<<<",
__func__, result, (int) isActive());
return NULL;
}
It continuously requests audio data from the application, and the obtained audio data is saved in a buffer and then written to the audio stream.
The callback loop for collecting audio streams (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternalCapture.cpp ) is as follows:
void *AudioStreamInternalCapture::callbackLoop() {
aaudio_result_t result = AAUDIO_OK;
aaudio_data_callback_result_t callbackResult = AAUDIO_CALLBACK_RESULT_CONTINUE;
if (!isDataCallbackSet()) return NULL;
// result might be a frame count
while (mCallbackEnabled.load() && isActive() && (result >= 0)) {
// Read audio data from stream.
int64_t timeoutNanos = calculateReasonableTimeout(mCallbackFrames);
// This is a BLOCKING READ!
result = read(mCallbackBuffer.get(), mCallbackFrames, timeoutNanos);
if ((result != mCallbackFrames)) {
ALOGE("callbackLoop: read() returned %d", result);
if (result >= 0) {
// Only read some of the frames requested. Must have timed out.
result = AAUDIO_ERROR_TIMEOUT;
}
maybeCallErrorCallback(result);
break;
}
// Call application using the AAudio callback interface.
callbackResult = maybeCallDataCallback(mCallbackBuffer.get(), mCallbackFrames);
if (callbackResult == AAUDIO_CALLBACK_RESULT_STOP) {
ALOGD("%s(): callback returned AAUDIO_CALLBACK_RESULT_STOP", __func__);
result = systemStopInternal();
break;
}
}
ALOGD("callbackLoop() exiting, result = %d, isActive() = %d",
result, (int) isActive());
return NULL;
}
It continuously reads data from the audio stream and then passes it to the application through the callback function.
AudioStreamThe process of creating a thread (located in frameworks/av/media/libaaudio/src/core/AudioStream.cpp ) is as follows:
void* AudioStream::wrapUserThread() {
void* procResult = nullptr;
mThreadRegistrationResult = registerThread();
if (mThreadRegistrationResult == AAUDIO_OK) {
// Run callback loop. This may take a very long time.
procResult = mThreadProc(mThreadArg);
mThreadRegistrationResult = unregisterThread();
}
return procResult;
}
// This is the entry point for the new thread created by createThread_l().
// It converts the 'C' function call to a C++ method call.
static void* AudioStream_internalThreadProc(void* threadArg) {
AudioStream *audioStream = (AudioStream *) threadArg;
// Prevent the stream from being deleted while being used.
// This is just for extra safety. It is probably not needed because
// this callback should be joined before the stream is closed.
android::sp<AudioStream> protectedStream(audioStream);
// Balance the incStrong() in createThread_l().
protectedStream->decStrong(nullptr);
return protectedStream->wrapUserThread();
}
// This is not exposed in the API.
// But it is still used internally to implement callbacks for MMAP mode.
aaudio_result_t AudioStream::createThread_l(int64_t periodNanoseconds,
aaudio_audio_thread_proc_t threadProc,
void* threadArg)
{
if (mHasThread) {
ALOGD("%s() - previous thread was not joined, join now to be safe", __func__);
joinThread_l(nullptr);
}
if (threadProc == nullptr) {
return AAUDIO_ERROR_NULL;
}
// Pass input parameters to the background thread.
mThreadProc = threadProc;
mThreadArg = threadArg;
setPeriodNanoseconds(periodNanoseconds);
mHasThread = true;
// Prevent this object from getting deleted before the thread has a chance to create
// its strong pointer. Assume the thread will call decStrong().
this->incStrong(nullptr);
int err = pthread_create(&mThread, nullptr, AudioStream_internalThreadProc, this);
if (err != 0) {
android::status_t status = -errno;
ALOGE("%s() - pthread_create() failed, %d", __func__, status);
this->decStrong(nullptr); // Because the thread won't do it.
mHasThread = false;
return AAudioConvert_androidToAAudioResult(status);
} else {
// TODO Use AAudioThread or maybe AndroidThread
// Name the thread with an increasing index, "AAudio_#", for debugging.
static std::atomic<uint32_t> nextThreadIndex{1};
char name[16]; // max length for a pthread_name
uint32_t index = nextThreadIndex++;
// Wrap the index so that we do not hit the 16 char limit
// and to avoid hard-to-read large numbers.
index = index % 100000; // arbitrary
snprintf(name, sizeof(name), "AAudio_%u", index);
err = pthread_setname_np(mThread, name);
ALOGW_IF((err != 0), "Could not set name of AAudio thread. err = %d", err);
return AAUDIO_OK;
}
}
AudioStreamBy pthread_create()creating a thread, performing the task AudioStream_internalThreadProc, and then setting the thread's name. After the newly created thread starts running, it first registers the current thread, and then runs the tasks thrown in when the thread was created, which is the callback loop of the AAudio audio stream. Registering the current thread will send thread information to media.aaudiothe service (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternal.cpp ):
aaudio_result_t AudioStreamInternal::registerThread() {
if (mServiceStreamHandle == AAUDIO_HANDLE_INVALID) {
ALOGW("%s() mServiceStreamHandle invalid", __func__);
return AAUDIO_ERROR_INVALID_STATE;
}
return mServiceInterface.registerAudioThread(mServiceStreamHandle,
gettid(),
getPeriodNanoseconds());
}
Although AudioStreamthe period duration is passed when requesting thread creation, periodic execution is not AudioStreamhandled in .
In the synchronous mode of audio stream data transfer, the application writes data to the AAudio audio stream to play the audio data. The interface implementation for writing audio data to the audio stream (located in frameworks/av/media/libaaudio/src/core/AAudioAudio.cpp ) is as follows:
AAUDIO_API aaudio_result_t AAudioStream_write(AAudioStream* stream,
const void *buffer,
int32_t numFrames,
int64_t timeoutNanoseconds)
{
AudioStream *audioStream = convertAAudioStreamToAudioStream(stream);
if (buffer == nullptr) {
return AAUDIO_ERROR_NULL;
}
// Don't allow writes when playing with a callback.
if (audioStream->isDataCallbackActive()) {
// A developer requested this warning because it would have saved lots of debugging.
ALOGW("%s() - Cannot write to a callback stream when running.", __func__);
return AAUDIO_ERROR_INVALID_STATE;
}
if (numFrames < 0) {
return AAUDIO_ERROR_ILLEGAL_ARGUMENT;
} else if (numFrames == 0) {
return 0;
}
aaudio_result_t result = audioStream->write(buffer, numFrames, timeoutNanoseconds);
return result;
}
This function checks the parameters and whether the audio stream is working in synchronous mode, and then write()writes the audio data through the operation of the audio stream object, the same operation used in the callback loop of asynchronous mode. write()The operation is implemented by a specific audio stream. For the audio playback stream, AudioStreamInternalPlay::write()the function is defined (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternalPlay.cpp ) as follows:
aaudio_result_t AudioStreamInternalPlay::write(const void *buffer, int32_t numFrames,
int64_t timeoutNanoseconds) {
return processData((void *)buffer, numFrames, timeoutNanoseconds);
}
AudioStreamInternalPlay::write()By AudioStreamInternal::processData()executing the data processing loop and performing the necessary wait (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternal.cpp ) until all data has been processed or the maximum wait time has expired:
aaudio_result_t AudioStreamInternal::processData(void *buffer, int32_t numFrames,
int64_t timeoutNanoseconds)
{
const char * traceName = "aaProc";
const char * fifoName = "aaRdy";
ATRACE_BEGIN(traceName);
if (ATRACE_ENABLED()) {
int32_t fullFrames = mAudioEndpoint->getFullFramesAvailable();
ATRACE_INT(fifoName, fullFrames);
}
aaudio_result_t result = AAUDIO_OK;
int32_t loopCount = 0;
uint8_t* audioData = (uint8_t*)buffer;
int64_t currentTimeNanos = AudioClock::getNanoseconds();
const int64_t entryTimeNanos = currentTimeNanos;
const int64_t deadlineNanos = currentTimeNanos + timeoutNanoseconds;
int32_t framesLeft = numFrames;
// Loop until all the data has been processed or until a timeout occurs.
while (framesLeft > 0) {
// The call to processDataNow() will not block. It will just process as much as it can.
int64_t wakeTimeNanos = 0;
aaudio_result_t framesProcessed = processDataNow(audioData, framesLeft,
currentTimeNanos, &wakeTimeNanos);
if (framesProcessed < 0) {
result = framesProcessed;
break;
}
framesLeft -= (int32_t) framesProcessed;
audioData += framesProcessed * getBytesPerFrame();
// Should we block?
if (timeoutNanoseconds == 0) {
break; // don't block
} else if (wakeTimeNanos != 0) {
if (!mAudioEndpoint->isFreeRunning()) {
// If there is software on the other end of the FIFO then it may get delayed.
// So wake up just a little after we expect it to be ready.
wakeTimeNanos += mWakeupDelayNanos;
}
currentTimeNanos = AudioClock::getNanoseconds();
int64_t earliestWakeTime = currentTimeNanos + mMinimumSleepNanos;
// Guarantee a minimum sleep time.
if (wakeTimeNanos < earliestWakeTime) {
wakeTimeNanos = earliestWakeTime;
}
if (wakeTimeNanos > deadlineNanos) {
// If we time out, just return the framesWritten so far.
// TODO remove after we fix the deadline bug
ALOGW("processData(): entered at %lld nanos, currently %lld",
(long long) entryTimeNanos, (long long) currentTimeNanos);
ALOGW("processData(): TIMEOUT after %lld nanos",
(long long) timeoutNanoseconds);
ALOGW("processData(): wakeTime = %lld, deadline = %lld nanos",
(long long) wakeTimeNanos, (long long) deadlineNanos);
ALOGW("processData(): past deadline by %d micros",
(int)((wakeTimeNanos - deadlineNanos) / AAUDIO_NANOS_PER_MICROSECOND));
mClockModel.dump();
mAudioEndpoint->dump();
break;
}
if (ATRACE_ENABLED()) {
int32_t fullFrames = mAudioEndpoint->getFullFramesAvailable();
ATRACE_INT(fifoName, fullFrames);
int64_t sleepForNanos = wakeTimeNanos - currentTimeNanos;
ATRACE_INT("aaSlpNs", (int32_t)sleepForNanos);
}
AudioClock::sleepUntilNanoTime(wakeTimeNanos);
currentTimeNanos = AudioClock::getNanoseconds();
}
}
if (ATRACE_ENABLED()) {
int32_t fullFrames = mAudioEndpoint->getFullFramesAvailable();
ATRACE_INT(fifoName, fullFrames);
}
// return error or framesProcessed
(void) loopCount;
ATRACE_END();
return (result < 0) ? result : numFrames - framesLeft;
}
AudioStreamInternal::processData()The data processing loop processes data through its subclasses processDataNow(). For playing audio streams, processDataNow()it performs the action of writing data, and for collecting audio streams, processDataNow()it performs the action of reading data. processDataNow()The execution does not block, it just processes as much data as possible.
The function definition for playing audio streams AudioStreamInternalPlay::processDataNow()(located in frameworks/av/media/libaaudio/src/client/AudioStreamInternalPlay.cpp ) is as follows:
aaudio_result_t AudioStreamInternalPlay::processDataNow(void *buffer, int32_t numFrames,
int64_t currentNanoTime, int64_t *wakeTimePtr) {
aaudio_result_t result = processCommands();
if (result != AAUDIO_OK) {
return result;
}
const char *traceName = "aaWrNow";
ATRACE_BEGIN(traceName);
if (mClockModel.isStarting()) {
// Still haven't got any timestamps from server.
// Keep waiting until we get some valid timestamps then start writing to the
// current buffer position.
ALOGV("%s() wait for valid timestamps", __func__);
// Sleep very briefly and hope we get a timestamp soon.
*wakeTimePtr = currentNanoTime + (2000 * AAUDIO_NANOS_PER_MICROSECOND);
ATRACE_END();
return 0;
}
// If we have gotten this far then we have at least one timestamp from server.
// If a DMA channel or DSP is reading the other end then we have to update the readCounter.
if (mAudioEndpoint->isFreeRunning()) {
// Update data queue based on the timing model.
int64_t estimatedReadCounter = mClockModel.convertTimeToPosition(currentNanoTime);
// ALOGD("AudioStreamInternal::processDataNow() - estimatedReadCounter = %d", (int)estimatedReadCounter);
mAudioEndpoint->setDataReadCounter(estimatedReadCounter);
}
if (mNeedCatchUp.isRequested()) {
// Catch an MMAP pointer that is already advancing.
// This will avoid initial underruns caused by a slow cold start.
// We add a one burst margin in case the DSP advances before we can write the data.
// This can help prevent the beginning of the stream from being skipped.
advanceClientToMatchServerPosition(getFramesPerBurst());
mNeedCatchUp.acknowledge();
}
// If the read index passed the write index then consider it an underrun.
// For shared streams, the xRunCount is passed up from the service.
if (mAudioEndpoint->isFreeRunning() && mAudioEndpoint->getFullFramesAvailable() < 0) {
mXRunCount++;
if (ATRACE_ENABLED()) {
ATRACE_INT("aaUnderRuns", mXRunCount);
}
}
// Write some data to the buffer.
//ALOGD("AudioStreamInternal::processDataNow() - writeNowWithConversion(%d)", numFrames);
int32_t framesWritten = writeNowWithConversion(buffer, numFrames);
//ALOGD("AudioStreamInternal::processDataNow() - tried to write %d frames, wrote %d",
// numFrames, framesWritten);
if (ATRACE_ENABLED()) {
ATRACE_INT("aaWrote", framesWritten);
}
// Sleep if there is too much data in the buffer.
// Calculate an ideal time to wake up.
if (wakeTimePtr != nullptr
&& (mAudioEndpoint->getFullFramesAvailable() >= getBufferSize())) {
// By default wake up a few milliseconds from now. // TODO review
int64_t wakeTime = currentNanoTime + (1 * AAUDIO_NANOS_PER_MILLISECOND);
aaudio_stream_state_t state = getState();
//ALOGD("AudioStreamInternal::processDataNow() - wakeTime based on %s",
// AAudio_convertStreamStateToText(state));
switch (state) {
case AAUDIO_STREAM_STATE_OPEN:
case AAUDIO_STREAM_STATE_STARTING:
if (framesWritten != 0) {
// Don't wait to write more data. Just prime the buffer.
wakeTime = currentNanoTime;
}
break;
case AAUDIO_STREAM_STATE_STARTED:
{
// Sleep until the readCounter catches up and we only have
// the getBufferSize() frames of data sitting in the buffer.
int64_t nextReadPosition = mAudioEndpoint->getDataWriteCounter() - getBufferSize();
wakeTime = mClockModel.convertPositionToTime(nextReadPosition);
}
break;
default:
break;
}
*wakeTimePtr = wakeTime;
}
ATRACE_END();
return framesWritten;
}
This function first calls to processCommands()process media.aaudiothe command sent by the service, then mainly executes writeNowWithConversion()writing audio data, and then calculates the waiting time and returns. AudioStreamInternalPlay::writeNowWithConversion()The function takes a piece of memory from the fifo we introduced earlier, processes the audio data, writes the processed audio data to the memory area, and moves the write counter:
aaudio_result_t AudioStreamInternalPlay::writeNowWithConversion(const void *buffer,
int32_t numFrames) {
WrappingBuffer wrappingBuffer;
uint8_t *byteBuffer = (uint8_t *) buffer;
int32_t framesLeft = numFrames;
mAudioEndpoint->getEmptyFramesAvailable(&wrappingBuffer);
// Write data in one or two parts.
int partIndex = 0;
while (framesLeft > 0 && partIndex < WrappingBuffer::SIZE) {
int32_t framesToWrite = framesLeft;
int32_t framesAvailable = wrappingBuffer.numFrames[partIndex];
if (framesAvailable > 0) {
if (framesToWrite > framesAvailable) {
framesToWrite = framesAvailable;
}
int32_t numBytes = getBytesPerFrame() * framesToWrite;
mFlowGraph.process((void *)byteBuffer,
wrappingBuffer.data[partIndex],
framesToWrite);
byteBuffer += numBytes;
framesLeft -= framesToWrite;
} else {
break;
}
partIndex++;
}
int32_t framesWritten = numFrames - framesLeft;
mAudioEndpoint->advanceWriteIndex(framesWritten);
return framesWritten;
}
In the synchronous mode of audio stream data transfer, the application reads data from the audio stream of AAudio to obtain the collected audio data. The interface implementation for reading data from the audio stream (located in frameworks/av/media/libaaudio/src/core/AAudioAudio.cpp ) is as follows:
AAUDIO_API aaudio_result_t AAudioStream_read(AAudioStream* stream,
void *buffer,
int32_t numFrames,
int64_t timeoutNanoseconds)
{
AudioStream *audioStream = convertAAudioStreamToAudioStream(stream);
if (buffer == nullptr) {
return AAUDIO_ERROR_NULL;
}
if (numFrames < 0) {
return AAUDIO_ERROR_ILLEGAL_ARGUMENT;
} else if (numFrames == 0) {
return 0;
}
aaudio_result_t result = audioStream->read(buffer, numFrames, timeoutNanoseconds);
return result;
}
This function checks the parameters and then read()reads the audio data through the operation of the audio stream object, the same operation used in the callback loop in asynchronous mode. read()The operation is implemented by a specific audio stream. For the audio collection stream, AudioStreamInternalCapture::read()the function is defined (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternalCapture.cpp ) as follows:
aaudio_result_t AudioStreamInternalCapture::read(void *buffer, int32_t numFrames,
int64_t timeoutNanoseconds)
{
return processData(buffer, numFrames, timeoutNanoseconds);
}
Like AudioStreamInternalPlay::write(), the data processing loop is also executed here AudioStreamInternal::processData()and the necessary waiting is performed until all data is processed or the maximum waiting time is reached. AudioStreamInternalCaptureThe processDataNow()definition (located in frameworks/av/media/libaaudio/src/client/AudioStreamInternalCapture.cpp ) is as follows:
// Read as much data as we can without blocking.
aaudio_result_t AudioStreamInternalCapture::processDataNow(void *buffer, int32_t numFrames,
int64_t currentNanoTime, int64_t *wakeTimePtr) {
aaudio_result_t result = processCommands();
if (result != AAUDIO_OK) {
return result;
}
const char *traceName = "aaRdNow";
ATRACE_BEGIN(traceName);
if (mClockModel.isStarting()) {
// Still haven't got any timestamps from server.
// Keep waiting until we get some valid timestamps then start writing to the
// current buffer position.
ALOGD("processDataNow() wait for valid timestamps");
// Sleep very briefly and hope we get a timestamp soon.
*wakeTimePtr = currentNanoTime + (2000 * AAUDIO_NANOS_PER_MICROSECOND);
ATRACE_END();
return 0;
}
// If we have gotten this far then we have at least one timestamp from server.
if (mAudioEndpoint->isFreeRunning()) {
//ALOGD("AudioStreamInternalCapture::processDataNow() - update remote counter");
// Update data queue based on the timing model.
// Jitter in the DSP can cause late writes to the FIFO.
// This might be caused by resampling.
// We want to read the FIFO after the latest possible time
// that the DSP could have written the data.
int64_t estimatedRemoteCounter = mClockModel.convertLatestTimeToPosition(currentNanoTime);
// TODO refactor, maybe use setRemoteCounter()
mAudioEndpoint->setDataWriteCounter(estimatedRemoteCounter);
}
// This code assumes that we have already received valid timestamps.
if (mNeedCatchUp.isRequested()) {
// Catch an MMAP pointer that is already advancing.
// This will avoid initial underruns caused by a slow cold start.
advanceClientToMatchServerPosition();
mNeedCatchUp.acknowledge();
}
// If the capture buffer is full beyond capacity then consider it an overrun.
// For shared streams, the xRunCount is passed up from the service.
if (mAudioEndpoint->isFreeRunning()
&& mAudioEndpoint->getFullFramesAvailable() > mAudioEndpoint->getBufferCapacityInFrames()) {
mXRunCount++;
if (ATRACE_ENABLED()) {
ATRACE_INT("aaOverRuns", mXRunCount);
}
}
// Read some data from the buffer.
//ALOGD("AudioStreamInternalCapture::processDataNow() - readNowWithConversion(%d)", numFrames);
int32_t framesProcessed = readNowWithConversion(buffer, numFrames);
//ALOGD("AudioStreamInternalCapture::processDataNow() - tried to read %d frames, read %d",
// numFrames, framesProcessed);
if (ATRACE_ENABLED()) {
ATRACE_INT("aaRead", framesProcessed);
}
// Calculate an ideal time to wake up.
if (wakeTimePtr != nullptr && framesProcessed >= 0) {
// By default wake up a few milliseconds from now. // TODO review
int64_t wakeTime = currentNanoTime + (1 * AAUDIO_NANOS_PER_MILLISECOND);
aaudio_stream_state_t state = getState();
//ALOGD("AudioStreamInternalCapture::processDataNow() - wakeTime based on %s",
// AAudio_convertStreamStateToText(state));
switch (state) {
case AAUDIO_STREAM_STATE_OPEN:
case AAUDIO_STREAM_STATE_STARTING:
break;
case AAUDIO_STREAM_STATE_STARTED:
{
// When do we expect the next write burst to occur?
// Calculate frame position based off of the readCounter because
// the writeCounter might have just advanced in the background,
// causing us to sleep until a later burst.
int64_t nextPosition = mAudioEndpoint->getDataReadCounter() + getFramesPerBurst();
wakeTime = mClockModel.convertPositionToLatestTime(nextPosition);
}
break;
default:
break;
}
*wakeTimePtr = wakeTime;
}
ATRACE_END();
return framesProcessed;
}
This function also first calls the processCommands()processing media.aaudiocommand sent by the service, and then mainly executes readNowWithConversion()reading the audio data, and then calculates the waiting time and returns. AudioStreamInternalCapture::readNowWithConversion()The function takes a piece of memory from the fifo we introduced earlier, copies the collected audio data from it, and moves the read counter:
aaudio_result_t AudioStreamInternalCapture::readNowWithConversion(void *buffer,
int32_t numFrames) {
// ALOGD("readNowWithConversion(%p, %d)",
// buffer, numFrames);
WrappingBuffer wrappingBuffer;
uint8_t *destination = (uint8_t *) buffer;
int32_t framesLeft = numFrames;
mAudioEndpoint->getFullFramesAvailable(&wrappingBuffer);
// Read data in one or two parts.
for (int partIndex = 0; framesLeft > 0 && partIndex < WrappingBuffer::SIZE; partIndex++) {
int32_t framesToProcess = framesLeft;
const int32_t framesAvailable = wrappingBuffer.numFrames[partIndex];
if (framesAvailable <= 0) break;
if (framesToProcess > framesAvailable) {
framesToProcess = framesAvailable;
}
const int32_t numBytes = getBytesPerFrame() * framesToProcess;
const int32_t numSamples = framesToProcess * getSamplesPerFrame();
const audio_format_t sourceFormat = getDeviceFormat();
const audio_format_t destinationFormat = getFormat();
memcpy_by_audio_format(destination, destinationFormat,
wrappingBuffer.data[partIndex], sourceFormat, numSamples);
destination += numBytes;
framesLeft -= framesToProcess;
}
int32_t framesProcessed = numFrames - framesLeft;
mAudioEndpoint->advanceReadIndex(framesProcessed);
//ALOGD("readNowWithConversion() returns %d", framesProcessed);
return framesProcessed;
}
The key operations of audio streaming are implemented roughly like this.
Done.