AAOS audio dynamic routing

basic concept

Android manages sounds from Android applications, controls these applications at the same time, and routes sounds to output devices in HAL according to their sound types: the entire framework is as shown below

• logical sound stream

Marked using the audio attribute, an application can have one or more logical sound streams that emit audio data through standard Android APIs (such as AudioManager for controlling focus or MediaPlayer for online playback). Application sound streams are associated with AudioAttributes. The logical sound stream is sent through the AudioService and routed to one (and only one) available physical output sound stream.

• output device

At the audio HAL level, the device type AUDIO_DEVICE_OUT_BUSprovides a generic output device for use in automotive audio systems. Bus devices support addressable ports (where each port is an endpoint of a physical sound stream)

Assignment of audio contexts to output devices is car_audio_configuration.xmldone via .

Car Audio Profile

Tags related to the car_audio_configuration.xml file

zone : Define the relevant information of each zone, corresponding to each partition, such as main area, rear seat area 1, rear seat area 2, etc.

primary zone : Primary audio zone. By default, all audio will be routed to the primary audio zone. There can only be one primary audio area, which is defined in xml through isPrimary="true".

zoneId : Audio zone ID, used to identify different audio zones. The audioZoneId of the main audio zone is always zero.

occupantZoneId : Passenger zone ID, used to configure routing based on user ID. For example, if you want each user to use a different audio zone, you need to configure it as shown in the example below. In the following example, the main audio area ID is 0, the passenger area ID is 0, the auxiliary audio area ID is 1, and the passenger area ID is 1. Among them, the occupiedZoneId needs the corresponding display to be valid.

<audioZoneConfiguration version="2.0">  
    <zone name="primary zone" isPrimary="true" occupantZoneId="0">    ...   
    </zone>   
    <zone name="rear seat zone" audioZoneId="1" occupantZoneId="1">    ...  
    </zone>  
    </zones>
</audioZoneConfiguration>

Mapping relationship between audioZoneId and occupiedZoneId

1. The audioZoneId of the main audio zone is always zero

2. The numbers of audioZoneId and occupiedZoneId cannot be repeated

3. There can only be a one-to-one mapping between audioZoneId and occupiedZoneId

group : Volume group, used to apply the same gain to all devices associated with the corresponding volume group. It is responsible for managing the volume of a series of devices in the audio area. Within each volume group, the volume can be controlled individually and the resulting gain can be configured on the associated device for application via the corresponding onboard amplifier. Volume settings are persisted for the user and loaded when the user logs in.

device : the address of the device, the device associated with the group

context : audio context, used in carAudioService to define routing, volume groups and audio focus management.

External way to configure audio area

• Configure by applying UID.
  First obtain the zoneId that needs to be configured, and then use the CarAudioManager#setZoneIdForUid API to set the audio routing.

• Configuring by User ID The default way
  CarAudioService will use the mapping between the car's passenger area and the audio area to define routing based on user ID. This way, the in-car audio service will be notified when the user logs into the passenger area. Upon receiving this signal, the system automatically configures audio focus management and routing for all audio zones.

• Routing using setPreferredDevice

  Android 11 also adds the following API to query the output device associated with each audio zone: CarAudioManager#getOutputDeviceForUsage(int zoneId, int usage). This API can be used to query the output device used for a specific audio zone and audio property usage. In this way, first-party applications can use the player's setPreferredDevice API to route audio to different audio zones. The getOutputDeviceForUsage API requires the use of PERMISSION_CAR_CONTROL_AUDIO_SETTINGS and is a system API. The following example shows how to find the media device used for a specific audio zone and route to the device using the setPreferredDevice API.

  audioZoneId = ... ;
  mediaDeviceInfo = mCarAudioManager.getOutputDeviceForUsage(audioZoneId, AudioAttributes.USAGE_MEDIA);
  mPlayer.setPreferredDevice(mediaDeviceInfo);
  

Vehicle audio service configuration routing process

• Parse xml and store it in CarAudioZone

1. Read car_audio_configuration.xml and parse it in the order of tags defined in xml
2. Parse all zones under the zones tag, confirm that there is only one primary zone, and store the zone in carAudioZones
3. Parse the audioZoneId and occupiedZoneId corresponding to the zone tag, ensure that the occupiedZontId is unique and stored in mZoneIdToOccupantZoneIdMapping
4. Create CarAudioZone based on the resolved zone name and id, and start parsing the volumeGroup
5. Parse the groupID tag. Each time a group is parsed, the corresponding groupID is increased by one. At the same time, it starts to parse the group's nested device and context.
6. Parse the device tag under groupID. Device corresponds to the address name. Make sure the bus address is in the output device. (How to confirm that a framework is involved)
7. Parse the context corresponding to the device, then obtain deviceinfo according to the address of the device, and then set the context corresponding to deviceInfo.
8. Finally, all the information parsed to group, device, and context will be stored in CarAudioZone.

• Synchronizing the gain-related information of each zone
  is achieved by calling hal adev_set_audio_port_config. Currently, this hal layer is not implemented. The gain of the zone is actually set by setting each group.

• Set dynamic routing for audio

  1. Traverse all zones. For each zone, traverse the volumeGroup in the zone.
  2. For volumeGroup, all addresses are traversed, and for addrss, all contexts are traversed.
  3. Obtain the usage of AudioAttributes for all contexts. One context may correspond to multiple usages. Create an AudioAttributes for each usage, and then add the AudioAttributes and device-related settings to AudioMix.

What AAOS provides to the outside is USAGE, and the above process creates a mapping relationship of address (device)–>context–>usage. For applications, after specifying AttributeUsages where the sound stream is used, carAudioService will find the corresponding device to be output based on the mapping relationship. What context actually corresponds to is the usage array of AudioAttributes (one context corresponds to a usage array)

AAOS mainly uses AudioAttributes.AttributeUsages for routing, volume adjustment and focus management. Usage is used to express the "reason" for playing a sound stream. Therefore, all sound streams and audio focus requests should specify a usage for their audio playback. If not set explicitly when building the AudioAttributes object, usage will default to USAGE_UNKOWN. While this usage is currently treated the same as USAGE_MEDIA, this behavior should not be relied upon for media playback.

Get available output devices in the framework

carAudioService obtains the output devices available for the device by calling AudioManager getDevice

The available devices are defined in audio_policy_configuration.xml. The device defined by the attachDevice tag is also defined in device_port, and the device is opened through the hal layer. The device that can be opened successfully is an available output device.

• audio_policy_configuration.xml parsing

  1. The module tag corresponds to HWModule, which corresponds to the implementation of different hal libraries, such as primary, usb, etc. The interfaces of different hal libraries are the same. The upper layer is called through the HIDL interface.

  2. MixProt is understood as a stream, and there is a corresponding profile. The format, sampling rate, and mask are configured in the profile. According to type, it is divided into input and output streams corresponding to the class IOProfile. Among them, mSupportedDevices represents

     当前流支持的设备集合；  
     * 如果是sink输入流，查找规则如下：  
     * 1. 遍历其父类的成员mRoutes，因为是输入流，所以遍历mRoute集合中sink为自己的route，也就是找有哪些源source设备把数据传给自己。  
     * 2. 找到route后，根据route中source保存的对象，且对象type是AUDIO_PORT_TYPE_DEVICE类型(就是devicesPort标签对应的实体类DeviceDescriptor)  
     * 3. 把DeviceDescriptor保存在集合中，保存在以下mSupportedDevices中，作为其支持的设备；
     * 输出流，同理；最终的结果就是：
     * 作为输出流source，mSupportedDevices保存此流可以输出到对应的device，stream -> device 
     * 作为输入流sink，mSupportedDevices保存了其他device能输出数据到此流，  device -> stream
     

  3. DevicePort tag The devicePort tag can be understood as a device. The device is also divided into output and input, but it is not divided by role like mixport, but by the keywords "IN" and "OUT" in type to divide the corresponding entity class DeviceDescriptor. The profile will also be parsed here.

  4. The label route route is a route that connects deviceport and mixport. Data is output from one stream to another device, or from one device to another stream.

Configuration example

• car_audio_configuration.xml

  The following xml configures 3 audio areas,

  The first one is the main audio area, its user ID is 0, the occupied area ID is 0, and 4 volume groups are defined. The devices in the volume group have the same audio gain.

  A volume group defines two output devices, and each output setting contains a set of audio contexts.

  The second one is rear seat area 1, whose user ID is 1, occupied area ID is 1, and defines a volume group. The audio group defines an output device and also contains a set of audio contexts.

  The third one is rear seat area 2. Its user ID is 2, its occupied area ID is 2, and it defines a volume group. The audio group defines an output device and also contains a group of audio contexts.

  If each audio zone and its respective occupied area ID are configured, then when the user logs in, such as when logging in to the main area, the audio stream of the corresponding application will use the devices of zone0. Then find the context according to the application setting AudioAttributes.AttributeUsages (the mapping of usages to context is as follows). For example, Usages is MEDIA, then find conetxt which is music, and context music finds the corresponding bus0_media_out device.

  If each audio zone and its respective user are not configured, the application needs to know the UID and ZONE ID.

  Then set the audio stream of the corresponding application to the device through the UID.

<?xml version="1.0" encoding="utf-8"?>
<carAudioConfiguration version="2">
    <zones>
        <zone name="primary zone" isPrimary="true" occupantZoneId="0">
            <volumeGroups>
                <group>
                    <device address="bus0_media_out">
                        <context context="music"/>
                        <context context="announcement"/>
                    </device>
                    <device address="bus6_notification_out">
                        <context context="notification"/>
                    </device>
                </group>
                <group>
                    <device address="bus1_navigation_out">
                        <context context="navigation"/>
                    </device>
                    <device address="bus2_voice_command_out">
                        <context context="voice_command"/>
                    </device>
                </group>
                <group>
                    <device address="bus4_call_out">
                        <context context="call"/>
                    </device>
                    <device address="bus3_call_ring_out">
                        <context context="call_ring"/>
                    </device>
                </group>
                <group>
                    <device address="bus5_alarm_out">
                        <context context="alarm"/>
                    </device>
                    <device address="bus7_system_sound_out">
                        <context context="system_sound"/>
                        <context context="emergency"/>
                        <context context="safety"/>
                        <context context="vehicle_status"/>
                    </device>
                </group>
            </volumeGroups>
        </zone>
        <zone name="rear seat zone 1" audioZoneId="1" occupantZoneId="1">
            <volumeGroups>
                <group>
                    <device address="bus100_audio_zone_1">
                        <context context="music"/>
                        <context context="navigation"/>
                        <context context="voice_command"/>
                        <context context="call_ring"/>
                        <context context="call"/>
                        <context context="alarm"/>
                        <context context="notification"/>
                        <context context="system_sound"/>
                        <context context="emergency"/>
                        <context context="safety"/>
                        <context context="vehicle_status"/>
                        <context context="announcement"/>
                    </device>
                </group>
            </volumeGroups>
        </zone>
        <zone name="rear seat zone 2"  audioZoneId="2" occupantZoneId="2">
            <volumeGroups>
                <group>
                    <device address="bus200_audio_zone_2">
                        <context context="music"/>
                        <context context="navigation"/>
                        <context context="voice_command"/>
                        <context context="call_ring"/>
                        <context context="call"/>
                        <context context="alarm"/>
                        <context context="notification"/>
                        <context context="system_sound"/>
                        <context context="emergency"/>
                        <context context="safety"/>
                        <context context="vehicle_status"/>
                        <context context="announcement"/>
                    </device>
                </group>
            </volumeGroups>
        </zone>
    </zones>
</carAudioConfiguration>

CarAudioContext	Associated AttributeUsages
MUSIC	UNKNOWN, GAME, MEDIA
NAVIGATION	ASSISTANCE_NAVIGATION_GUIDANCE
VOICE_COMMAND	ASSISTANT, ASSISTANCE_ACCESSIBILITY
CALL_RING	NOTIFICATION_RINGTONE
CALL	VOICE_COMMUNICATION, VOICE_COMMUNICATION_SIGNALING
ALARM	ALARM
NOTIFICATION	NOTIFICATION, NOTIFICATION_*
SYSTEM_SOUND	ASSISTANCE_SONIFICATION
EMERGENCY	EMERGENCY
SAFETY	SAFETY
VEHICLE_STATUS	VEHICLE_STATUS
ANNOUNCEMENT	ANNOUNCEMENT

Test Methods

• Summary of test methods

The upper layer only needs to verify whether the configured route is correct.

1. Use the kitchensink Audio page to test audio routing in different scenarios. It contains different usages and can select different zones. By playing different usage, and then checking the log to see if the corresponding device is found for output. For example, ASSISTANCE_NAVIGATION_GUIDANCE, the corresponding address is bus1_navigation_out. Then add log confirmation to the policy. The following log output indicates that the routing is successful.

   APM_AudioPolicyManager: setOutputDevices device {type:0x1000000,@:bus1_navigation_out} delayMs 0
   

3. For different userIDs, use the music application on different displays to check the routing devices to confirm whether different devices are routed in different areas. The music application can only test the case where the context is music. For example, with the current configuration above, music played on the main screen is routed to bus0_media_out for playback, while on the secondary screen it is routed to bus200_audio_zone_2 for playback.

• Related changes

1. car_audio_configuration.xml 添加zone，occupation，address，context
2. audio_policy_configuration.xml configures the device of address
3. Modify audioUseDynamicRouting to true in config.xml in overlay

Related questions

• How to configure audio zones

  Configure occupiedZoneId and zonID in car_audio_configuration.xml. These two correspond to one-to-one, and occupiedZoneId is used to map UserID and DisplayID. For example, car_audio_configuration.xml defines three occupiedZoneIds, which are 0, 1, and 2 respectively. Three audio area-related configurations are defined. It should be noted that if the displayPort=1 defined in config.xml config_occupant_display_mapping is invalid, then the audio zonId corresponding to the audio's occupiedZoneId of 1 is also invalid.

• How is the audio ZoneID associated with the UserID?

  The car audio service listens for the activation of the car account. When the car account is activated and the mapping relationship between the car members and the display is parsed, it will be called back to the car audio service. Here we will traverse the mapping array of zoneID and OccupantZoneId saved after parsing the in-vehicle audio configuration, take out zoneID and OccupantZoneId from the array, then obtain userID through OccupantZoneId from OccupantZoneService, and finally build a map of AudioZoneId and userID. At the same time, the routing is also updated in the audioPolicy, that is, the routing on the audiopolicy side also saves the userid information. When the application starts audio playback later, audioplicy will actively obtain the UserID and then match the route corresponding to the UserID.