Learning from Apple developers:AirPlay | Apple Developer Documentation
Airplay
AirPlay allows people to wirelessly stream media content from iOS, iPad, macOS and tvOS devices to AirPlay-enabled Apple TV, HomePod, as well as TVs and speakers.
Best Practices for Web Links
Prefer the media player provided by your system. The built-in media player provides a standard set of controls and supports features such as chapter navigation, subtitles, closed captions, and AirPlay streaming. It's also easy to implement, provides a consistent and familiar playback experience across the entire system, and adapts to the needs of most media applications. Only consider designing a custom video player if the player provided by the system cannot meet your application needs. For developer guidance, see AVPlayerViewController.
Provide content in the highest possible resolution. Your HTTP Live Streaming (HLS) playlist needs to include all available resolutions so people can experience your content at a resolution appropriate for the device they are using (AVFoundation automatically selects the resolution based on the device). If you don't include a range of resolutions, your content will look low quality when people stream it to a device that can play it at a higher resolution. For example, content that looks great on an iPhone at 720p will appear to be of poor quality when people use AirPlay to stream it to a 4K TV.
Only play what people expect. Avoid streaming content like background loops and short video experiences that only make sense within the context of the app itself. See usesExternalPlaybackWhileExternalScreenIsActive for developer guidance.
Supports AirPlay streaming and mirroring. Supporting both features simultaneously gives users maximum flexibility.
Support remote control events. When you do this, people can choose actions like play, pause, and fast forward on the lock screen, or through interactions with Siri or HomePod. For developer guidance, see Remote Command Center Events.
Don't stop playing when your app goes into the background or the device is locked. For example, people want a TV show started in your app to continue playing while they check their email or put their device to sleep. In this case, it's also important to avoid automatic mirroring, as people don't want to play other content on their devices without an explicit choice.
Don’t interrupt other apps’ playback unless your app starts playing immersive content. For example, if your app plays a video on startup or autoplays an inline video, only play that content locally on the device while allowing the current playback to continue. See ambient for developer guidance.
Let people use other parts of the app while playing. Your app needs to remain functional while AirPlay is active. If the user navigates away from the playback screen, ensure that other videos within the app do not start playing and interrupt the streaming content.
If necessary, provide a custom interface for controlling media playback. If you can't use the system-provided media player, you can create a custom media player to let people enter AirPlay in an intuitive way. If you need to do this, be sure to provide custom buttons that match the appearance and behavior of system-provided buttons, including different visual states that indicate when playback has started, is occurring, or is unavailable. In the custom control that launches AirPlay, use only the symbols provided by Apple and place the AirPlay icon correctly in the custom player - i.e. in the lower right corner (in iOS 16 and iPadOS 16 and later).
Add AirPlay support to your app
AirPlay enables you to wirelessly send content from your Apple device to your Apple TV or AirPlay-enabled speakers. AirPlay provides enhanced support for wireless audio distribution, including the ability to send content to multiple AirPlay-enabled speakers.
Identify the type of audio played by the application in page links
In iOS, tvOS, and watchOS, set the route sharing policy for audio sessions to .longform. Long-form audio is anything other than system sounds, such as music, audiobooks, or podcasts. This setting identifies the audio played by the application, such as the following example.
let audioSession = AVAudioSession.sharedInstance()
try audioSession.setCategory(.playback,
mode: .default,
policy: .longFormAudio)
Add an AirPlay pickerin
Add AVRoutePickerView to your view hierarchy to include an AirPlay picker in your app. The selector provides the user with a list of potential AirPlay devices they can use in your app. To control when the selector is displayed, use AVRouteDetector to identify the status of the route detector.
iOS, iPadOS, Mac Catalyst, tvOS
class AVRoutePickerView : UIViewmacOS
class AVRoutePickerView : NSViewWhen the user clicks the button, a window pops up showing all nearby AirPlay devices that can receive and play media. If your app prefers video content, devices that support video will appear in the list.
Configure text, color, and media preferences for buttons in page links
The following code example creates a view next to custom text:
HStack {
Text("Choose output device")
.font(.title)
.frame(maxWidth: .infinity, alignment: .center)
.fixedSize()
.padding(.leading)
if routeDetected {
DevicePickerView() // See implementation below.
.frame(width: 60, height: 60)
.padding(.trailing)
}
}
Your app configures the color scheme of the button and indicates whether your app likes video content, as shown in the following code:
struct DevicePickerView: UIViewRepresentable {
func makeUIView(context: Context) -> UIView {
let routePickerView = AVRoutePickerView()
// Configure the button's color.
routePickerView.delegate = context.coordinator
routePickerView.backgroundColor = UIColor.white
routePickerView.tintColor = UIColor.black
// Indicate whether your app prefers video content.
routePickerView.prioritizesVideoDevices = true
return routePickerView
Add media player
Use the API to customize your AirPlay integration with media players. Use MPRemoteCommandCenter to receive remote commands. If you use MPNowPlayingInfoCenter, you can notify the system metadata about audio tracks playing on the device.
Configure a fast streaming media page link application
Use one of two playback API sets that take advantage of AirPlay enhanced buffering:
For simple enhanced buffering, use AVPlayer or AVQueuePlayer. This works great for video content. For more information, see Implementing Simple Enhanced Buffering for Content.
For more flexible buffering enhancement, use AVSampleBufferAudioRenderer and AVSampleBufferRenderSynchronizer. This option is better for applications that need to control I/O, preprocess media data, or have DRM models that AVPlayer does not support. For more information, see Implementing Flexible Enhanced Buffering for Content.
HomeKit
link
HomeKit allows people to securely control connected accessories in their home using Siri or the Home app on iPhone, iPad, Apple Watch and Mac.
On iOS, the Home app also allows users to manage and configure accessories.
Your iOS, tvOS, or watchOS app can integrate with HomeKit (and by extension Home apps) to deliver custom or accessory-specific experiences. For example, you can:
Help people set up, name and organize their accessories
Allows fine-grained attachment configuration and control
Provide access to custom accessory functionality
Show people how to create powerful, hands-free automation
provide support
See HomeKit for developer guidance. If you are an MFi licensee, please visit the MFi Portal for guidance on naming and messaging for attachment packages.
Terminology and layout page link
HomeKit models a home as a hierarchy of objects and defines a vocabulary of terms that refer to them. The Home app uses the HomeKit object model and terminology to let people intuitively control accessories through voice, apps, and automation.
It's critical for your app to use the terminology and object model defined by HomeKit so that you can enhance people's understanding and make home automation approachable.
In the HomeKit model, the home object is the root of the hierarchy that contains all other objects, such as rooms, accessories, and areas. When there are multiple homes, each home is the root of a different hierarchy.
Acknowledge the layered model used by HomeKit. Even if your app doesn't organize accessories by room and zone in its UI, it can be useful to reference the HomeKit model when helping people set up or control accessories. People need to know where accessories are so they can use Siri and HomePod to control them, saying commands like "Siri, turn on the lights upstairs" or "It's dark in here." For more guidance,
Make it easier for people to find relevant HomeKit details for accessories. If your app focuses on accessories, don’t hide other HomeKit information, such as the accessory’s area or room, in a hard-to-find settings screen. Instead, consider making relevant HomeKit information easily available in the attachment details view.
Recognize that people can have more than one home. Even if your application does not support the concept of multiple households per user, consider providing relevant household information in the accessory details view.
Do not render duplicate home settings. If your app has a different angle for organizing a home, don't confuse people by asking them to reconfigure all or part of their home, or by showing duplicate settings views. Always follow the settings people have made in the Home app and find an intuitive way to present these details in your UI.
Home page link
HomeKit uses the word "home" to mean a physical home, office, or other location that people relate to. A person may have multiple houses.
Rooms
A room represents a physical room in the home. Rooms don't have attributes like size or location; they just have names that are meaningful to people, like "bedroom" or "office." When people assign accessories to a room, they can use voice commands like "Siri, turn on all lights except bedrooms," or "Siri, turn on kitchen and hallway lights."
Accessories, services and features page links
The term “accessory” refers to a physical, connected home accessory such as a ceiling fan, light, lock, or camera. HomeKit uses categories to represent an accessory type, such as a thermostat, fan, or light. Typically, accessory manufacturers assign each accessory to a category, but your app can help people make assignments if necessary. For example, a switch that connects a fan or light needs to be assigned to the same category as the accessory it controls.
The controllable properties of an accessory, such as a connected light switch, are called services. Some accessories provide multiple services. For example, a connected garage door could allow one to control the lights and door independently, or a connected outlet could support separate control of the top and bottom exits. Apps don't use the word "service" in the UI; instead, they use names that describe services, like garage door openers and ceiling fan lights. When people use Siri to control accessories in their home, they speak the name of the service, not the name of the accessory. For more guidance on naming, see Helping People Choose Useful Names.
Characteristics are controllable attributes of a service. For example, in a ceiling fan, the fan service might have a speed characteristic, and the light service might have a brightness characteristic. Apps don't use the word "property" in their UI; instead, they use terms that describe attributes, such as speed and brightness.
A service group represents a set of subordinate services that someone might wish to control as a unit. For example, if there is a floor lamp and two table lamps in a corner of a room, one might assign these three services to a service group called Reading Lamps. Doing this allows one to use a reading light service group to control these three lights independently of all the other lights in the room.
Links to action and scene pages
The term "action" refers to a change in a service characteristic, such as adjusting the speed of a fan or the brightness of a light. People and automation can initiate operations.
A scenario is a set of actions that control one or more services in one or more attachments. For example, one might create a movie time scene that lowers the shadows and dims the lights in the living room, or a good morning scene that turns on the lights, raises the shadows, and starts the coffee machine in the kitchen.
hint
The HomeKit API uses the term "action set" instead of "scene". In your application UI, always use the term scene.
Automated page links
Automation allows accessories to react to specific situations, such as when a person's location changes, when a specific time of day occurs, when another accessory turns on or off, or when a sensor detects something. For example, an automation device could turn on the lights in a home at sunset or when people arrive home.
Regional page link
A zone represents an area of a home that contains multiple rooms, such as upstairs or downstairs. Setting up a zone is optional, but doing so allows people to control multiple accessories at once. For example, assign all downstairs lights to a zone called Downstairs, allowing people to use voice commands like "Siri, turn off all the lights downstairs."
Set page link
Use the setup process provided by the system to give people a familiar experience. HomeKit's setup process is faster than traditional setup, as it lets users name accessories, join a network, pair with HomeKit, assign room and service categories, and specify favorites in just a few steps. Using the setup process provided by the system also allows your app to focus on promoting customization features to make your accessories unique. For developer guidance, see addAndSetupAccessories(with:completionHandler:).
Provide context to explain why access to people's Home data is needed. Create a purpose string that describes in one phrase why you are asking for permission to access the data, such as "Let you control this accessory through the Apple Home app and Siri on your Apple device."
Don't ask people to create accounts or provide personal information. Instead, follow HomeKit for any information you might need. If your app provides additional services that require an account, such as a cloud service, set Account Settings as optional and wait until after the initial HomeKit setup to provide it.
Respect people's setting choices. When people choose to use HomeKit to set up your accessories, don't force them to set up other platforms during the HomeKit setup process. A cross-platform setup experience prevents people from immediately using the accessory and can cause confusion by presenting too many ways to control the accessory.
Think carefully about how and when to deliver a custom accessory setup experience. Always start by presenting the setup process provided by the system. Then, after the basic functionality of the accessory is available, provide a customized post-installation experience that highlights your accessory’s unique capabilities and helps people get the most out of it. For example, a lighting manufacturer’s app helps people create personalized lighting scenes in their homes, using key colors scanned from photos in their library.
Siri interactive page link
HomeKit supports powerful hands-free control using voice commands. You can help people use Siri to quickly and efficiently interact with accessories, services, and areas in their home.
Provides voice command examples that demonstrate using Siri to control accessories during setup. Once people have finished setting up their new accessory, consider using the service name of their choice in a few example Siri phrases and encourage people to try it out.
Once you have it set up, consider teaching people more complex Siri commands. People may not be aware of the wide range of natural language phrases they can use to control their accessories using Siri and HomePod. Once set up, find useful places in the application to help people learn about these types of commands. For example, in the scene details view, you can tell people that you can say "Hey Siri, set 'movie time.'"
In addition to identifying the names of houses, rooms, areas, services and scenes, Siri can also use information such as accessory categories and characteristics to identify services. For example, when people use words like "brighter" or "darker," Siri can recognize that they're referring to a service that has brightness characteristics, even if they don't say the name of the service.
HomeKit enables your apps to coordinate and control home automation accessories from multiple vendors to present a coherent, user-centered interface.
This example shows a simplified version of the kind of app a HomeKit-enabled accessory manufacturer might offer. You use this app to configure and control a specific device—a garage door opener from the fictional manufacturer Kilgo Devices. The app offers minor and custom features not available in Apple's Home app. It provides a user experience consistent with the Home app's approach and terminology, but does not attempt to replicate every feature of the Home app.
Configure sample code projection page link
To be able to use HomeKit, you need to enable HomeKit functionality and include the NSHomeKitUsageDescription key in your app's Info. In this sample app, this feature is enabled and instructions for using it are provided.
Before building and running the application, perform the following steps:
Set a valid signing team in the target's General pane so that Xcode can create a profile containing HomeKit entitlements when you first build.
Download and install the Home Accessory Simulator (HAS) on your Mac to be able to emulate HomeKit-enabled accessories. See Testing Your App with the HomeKit Accessory Simulator.
Import the hasaccessory file bundled with the sample app to define the specific garage door opener accessories that the app controls. Select File > Import Accessory from the HAS menu. In the dialog box that appears, navigate to the Documentation folder of the downloaded sample code project and select Garage. hasaccessory file.
The import creates a single attachment with a hidden attachment information service (all attachments have it), and two user interaction services: one controls the garage door and the other controls the connected light bulb. Most of the relevant features are standard for their services. Only one item - the bulb's decay rate - is customizable. You can use HAS to inspect and operate all of these items.
Create a home page manager and get
You always use an instance of HMHomeManager as the root HomeKit object. Home Manager contains a set of homes, each with a set of accessories. The sample application defines a HomeStore class, used as a singleton, which holds a unique home manager for the application:
class HomeStore: NSObject {
/// A singleton that can be used anywhere in the app to access the home manager.
static var shared = HomeStore()
/// The one and only home manager that belongs to the home store singleton.
let homeManager = HMHomeManager()
/// A set of objects that want to receive home delegate callbacks.
var homeDelegates = Set<NSObject>()
/// A set of objects that want to receive accessory delegate callbacks.
var accessoryDelegates = Set<NSObject>()
}
You create an accessory list collection view controller to display a list of connected accessories. Because this root view controller is never released, it can safely assign itself as an HMHomeManagerDelegate protocol delegate:
HomeStore.shared.homeManager.delegate = self
The home manager tells its delegates when the home list changes, including the first time the home manager loads data from the HomeKit database during initialization. When this happens, the accessories list will reload to show the accessories in the home directory, or prompt the user to create a new directory if it does not exist.
You could extend the application to allow the user to choose among all known houses, instead of always selecting the main house. You can also allow users to add, delete, or rename homes, although users rarely perform these tasks and typically rely on the Homes app to complete them.
Add new accessory page link
The first time you run the app, the accessories list is empty because you don't have any accessories associated with Kilgo Devices. The app's UI has a + button on the navigation bar. Click this button to start searching for accessories on the local network. The button's tap handler calls home's addAndSetupAccessories(completionHandler:) method.
home?.addAndSetupAccessories(completionHandler: { error in
if let error = error {
print(error)
} else {
// Make no assumption about changes; just reload everything.
self.reloadData()
}
})
This provides the standard HomeKit UI for locating and adding new accessories to a given home. If successful, the completion handler refreshes the application's copy of the HomeKit data and redraws the display.
When your app enters the standard accessory association process (the same process used by the Home app), users need to follow these steps:
Scan or enter the HomeKit setup code for your new device. This code is packaged with the device or is available in the HAS display of the accessory.
Select from a list of devices on your network that don't have an existing HomeKit association. This includes both real and simulated devices that can access the LAN.
Wait for HomeKit to verify that the installation code entered in step 1 matches the device's code.
Assign a name and room to each service associated with the device. HomeKit provides a default name and space for each service, and a page per service that the user can accept or change.
Each item the user specified in step 4 will appear as an "accessory" in the Home app. However, in HomeKit these are HMService instances. They belong to an HMAccessory instance, which represents the physical device you selected in step 2. To maintain a consistent user experience with the Home application, the sample application UI (and the remainder of this article) also references each HMService instance as an attachment.
Only show accessory-related apps
Draw the display based on a copy of the data captured from HomeKit into the Kilgo service array:
var kilgoServices = [HMService]() // These are called "accessories" in the UI.
When the accessory list reloads, either because a new home is set up or because the accessory adding process completes, you populate the above array, filtering out HMAccessory instances from manufacturers other than Kilgo, and HMService instances that do not interact with the user Example. Additionally, when collecting a list, you can also request notification of changes to any corresponding characteristics, which are a single point of status and control for a given service:
for accessory in home.accessories.filter({ $0.manufacturer == "Kilgo Devices, Inc." }) {
accessory.delegate = HomeStore.shared
for service in accessory.services.filter({ $0.isUserInteractive }) {
kilgoServices.append(service)
// Ask for notifications from any characteristics that support them.
for characteristic in service.characteristics.filter({
$0.properties.contains(HMCharacteristicPropertySupportsEventNotification)
}) {
characteristic.enableNotification(true) { _ in }
}
}
}
Therefore, the display only shows accessories relevant to that specific application.
Customize common interactions for specific accessories
Accessories (such as light bulbs) have characteristics that the user controls or observes (such as power status, color temperature, brightness, etc.). Users usually care more about one of these features because they change or read that feature most often. This is the main feature and you should give users quick access to it. For a light bulb, the user usually wants to turn it on or off, so the power state is the main feature.
It's up to you to define the main characteristics of the accessories you control. You can return the type of the primary attribute by creating a computed property in the HMService extension:
var primaryControlCharacteristicType: String? {
switch kilgoServiceType {
case .lightBulb: return HMCharacteristicTypePowerState
case .garageDoor: return HMCharacteristicTypeTargetDoorState
case .unknown: return nil
}
}
This primary feature type is then used to locate and return features of that type:
var primaryControlCharacteristic: HMCharacteristic? {
return characteristics.first { $0.characteristicType == primaryControlCharacteristicType }
}
Kilgo Devices' light bulbs and garage doors are both binary. The light bulb is on or off. The target state of the door is open or closed. This makes it an interface where one toggle switch is enough to control all major features. You can implement this as a tap handler for each item in the attachment list collection view. When the user taps the accessory, read the current characteristic value and negate it.
func tap() {
if let characteristic = service?.primaryControlCharacteristic,
let value = characteristic.value as? Bool {
// Provide visual feedback that the item was tapped.
bounce()
// Write the new value to HomeKit.
characteristic.writeValue(!value) { error in
self.redrawState(error: error)
}
}
}
Writing involves network access, so when the writing is complete, HomeKit calls a completion handler. Use this opportunity to update the state of the interface, as shown in the code snippet above.
Enable custom configuration
When the user clicks the accessory's information button, the application displays detailed information about the accessory. In the detailed view, users can rename the accessory, assign it to a room, remove it from the home, and view device information such as firmware version. Users can also click "Settings" to display a list of secondary features for the accessory.
Control the user experience by presenting only relevant feature types. The KilgoService extension of HMService defines a computed property that limits the list of features that can be displayed to a curated list:
var displayableCharacteristics: [HMCharacteristic] {
let characteristicTypes = [HMCharacteristicTypePowerState,
HMCharacteristicTypeBrightness,
HMCharacteristicTypeHue,
HMCharacteristicTypeSaturation,
HMCharacteristicTypeTargetDoorState,
HMCharacteristicTypeCurrentDoorState,
HMCharacteristicTypeObstructionDetected,
HMCharacteristicTypeTargetLockMechanismState,
HMCharacteristicTypeCurrentLockMechanismState,
KilgoCharacteristicTypes.fadeRate.rawValue]
return characteristics.filter { characteristicTypes.contains($0.characteristicType) }
}
These are mostly HomeKit standard types, and all of them will work with Kilgo devices. In the same extension, there is also a custom type -rate—defined
enum KilgoCharacteristicTypes: String {
case fadeRate = "7E536242-341C-4862-BE90-272CE15BD633"
}
Characteristic types are stored as UUID strings. The value specified for the fade rate in the code matches the value found in the attachment simulator, which you can check in HAS. If you also make a real Kilgo device, the values it uses must match as well.