Method for obtaining iOS application element positioning through Appium and Accessibility Inspector

Locating elements using selectors on iOS mobile apps is a prerequisite for our UI automation testing on mobile.
However, because of the way app content is rendered in native iOS apps, the selectors we can use to target app elements are very different from web browser elements.
In this article, we'll look at 3 common ways to determine element selectors to locate elements in an iOS application for automated mobile UI testing.

XCTest framework and Appium XCUITest driver

For native iOS apps, app content is rendered through WKWebView (iOS 8.0 or later). The official XCTest framework developed by Apple provides a way to locate elements based on their rendered content, based on their element type and element properties in the WKWebView context. However, the XCTest framework only supports Swift and Objective-C, so it is not suitable for mobile UI automation testing when the iOS application is written in React Native instead of Swift.
If you already know or apply mobile UI automation testing, you may have heard of such an alternative: Appium XCUITest Driver (formerly known as UIAutomation Driver for iOS). The Appium XCUITest driver leverages Apple's XCTest framework and its underlying libraries to improve the automation of iOS applications and supports multiple languages including Java, Python, javascript, Ruby, C# and PHP. Next, we will use the Appium XCUITest driver as an example to see how we can inspect an iOS application and find element positioning.

Find element selectors for iOS mobile apps

The XCTest framework or the Appium XCUITest driver provide a way to programmatically interact with app elements, but we still need to manually identify element selectors and provide the selectors to our elements for UI automation testing. Since we have to use the iOS Simulator or a real iOS device to open the application, there is no built-in inspector on the device to provide us with this information, instead in various browsers, there are various developer tools available to provide this information.

Below are 3 methods that are commonly used to inspect native iOS application content and determine selectors to locate elements.
By the way, the sample iOS application we will use for demonstration is as follows. It's written in React Native and runs on the iOS Simulator.

(Screenshot of a sample react-native app running on the iOS simulator)

1. driver.getPageSource()

The Appium XCUITest driver provides a method: getPageSource(), which can directly print out the page source code of the currently rendered application page in the terminal console. This is a quick way to get a quick overview of all the elements in your app's page, and determine the element types and attributes that can be used to get element targeting.
Just insert the following code (Java code below) in the UI automation test:
System.out.println(driver.getPageSource());
Below is the page source code snippet of the sample application.

It can be seen that the printed page source code provides rich information about each element, including the type and attributes of the element, etc. We can use this information to determine the location of each element.
For example, for the Example Button button element, its page source code is:
<XCUIElementTypeButton type="XCUIElementTypeButton" name="Example Button" label="Example Button" enabled="true" visible="true" x="24" y= "431" width="327" height="38"/> </XCUIElementTypeOther>
We can easily determine its element positioning as:
driver.findElement(By.xpath("//XCUIElementTypeButton[@name='Example Button ']"))

2. Accessibility Inspector

Accessibility Inspector is a little-known but very useful tool developed by Apple to help developers improve the accessibility of iOS applications. The tool is bundled with Xcode, so Xcode needs to be installed on the Mac computer. Accessibility Inspector clarifies the accessibility features and issues of iOS applications by parsing and identifying each element on the iOS application and inspecting the element's type and accessibility tags. Since it provides detailed information about each element on the application, it can also be easily used to identify selectors for each element in automated UI tests.
Here is the information provided by the Accessibility Inspector for the Example Button button element:

(Screenshots of the sample app and the Accessibility Checker)

As you can see, the Accessibility Inspector shows that the Example Button button element has a Button type and an Example Button tag, so we can use them to locate the element:
driver.findElement(By.xpath("//XCUIElementTypeButton[@name='Example Button ']"))
The Accessibility Inspector isn't always easy to use at first when it comes to determining the exact element selector, but once you get familiar with how the Accessibility Inspector presents information about your application's elements, it's a quick and lightweight way to Class tools that get the job done with ease.

Practical examples:
1) Accessibility Inspector at the entry of Xcode: Xcode - Open Developer Tool - Accessibility Inspector

2) After connecting the device, click the start button to start using it.

3. Desktop App

Appium Desktop is a desktop application developed by the Appium team for Mac, Windows and Linux. It is a popular tool for many Appium developers to inspect iOS and Android application elements in mobile UI automation testing.
Here's what Appium Desktop displays about the Example Button button element:

(Screenshots of the sample app and Appium desktop)

It can be seen that the App Source in the middle part shows the complete hierarchical structure of the current application page, and the Selected Element in the right part shows detailed information and property list. What's more useful is that it also provides the best selector for locating the current element, namely accessibility id or xpath, both of which are very convenient to use.
Appium Desktop also offers other nice features such as direct clicking/tapping on selected elements, and refreshing the page source after applying page changes. You can refer to the official Appium Desktop code repository for more information.

Final Words
In summary, Appium Desktop is a popular and recommended tool for inspecting elements on iOS applications for element positioning. If you're new to writing UI automation tests, it's also useful to have a comprehensive view of your application and page elements. Once you're fairly familiar with the application and page structure, using driver.getPageSource() directly in code or using the Accessibility Inspector are good options for quickly locating elements.

Reference

1. 3 Ways to Find Element Selector for iOS App with Appium: https://stephenweixu.com/blog/3-ways-to-find-element-selector-for-ios-app-with-appium
2. iOS Test WebDriverAgent Introduction: https://testerhome.com/topics/4904 (use of WebDriverAgent Inspector)
3. Appium1.6 locates iOS elements and operation elements: https://www.cnblogs.com/meitian/p/7373460.html
4. Appium+Python3+iOS positioning elements: https://wangxiaoxi.cn/posts/appium-ios-2/ (the problem of slow element positioning)
5. uiautomator2 uses Python to test Android applications: https://www.cnblogs.com/ fnng/p/8486863.html (It is also mentioned that this method can obtain element positioning)