UI automated testing based on Appium

Why you need UI automation testing

Mobile APP is a complex system, and the coupling between different functions is very strong. It is difficult to guarantee the overall function only through unit testing. UI testing is an important part of mobile application development, but the execution speed is slow and there are a lot of repetitive workloads. In order to reduce these workloads and improve work efficiency, it is necessary to introduce sustainable integrated automated testing solutions.

Why choose Appium

Appium is an open source testing tool that can be used to test native applications and Web hybrid applications on Android/iOS/Windows.

In order to cope with the rapid iteration of mobile application functions, more and more apps adopt a hybrid mode, that is, some functions are handed over to the embedded Web page of the application. Appium can easily switch and test native applications or web pages embedded in the App, and has good support for Hybrid App.

Appium uses the testing framework provided by each platform itself, so there is no need to introduce third-party code or repackage applications.

platform	Test framework
Android 4.2+	UiAutomator/UiAutomator2 (default)
Android 2.3+	Instrumentation (provided by Selendroid)
iOS 9.3 and above	XCUITest
Below iOS 9.3	UIAutomation

Appium is open source on GitHub, is maintained frequently, and the community is relatively active. With the continuous efforts of the community, Appium can always maintain compatibility with the latest version of the mobile phone operating system and the official test framework, and its functions are becoming more and more complete, including basic log collection, screen recording, opencv-based image recognition, etc., and recently IOS 13/Android 10 support added by the version;
Appium supports finding elements through custom plug-ins. There are also third-party plug-ins that are available on GitHub, such as the icon recognition control example project based on artificial intelligence. You can also customize plug-ins to find page elements using image recognition, OCR, and other methods.

Use Cucumber to organize case

Appium supports a variety of programming languages, including Java, Python, etc., but the direct use of code to maintain the case is less readable and the learning cost is relatively high. The introduction of Cucumber can organize the case in a way closer to natural language. Cucumber is a tool that supports BDD (Behaviour-Driven Development). You can customize the grammar rule template and turn the steps described in text into steps that are executed by code. Because Cucumber and Java 8 are compatible with Chinese text encoding, you can customize the Chinese operation steps, which is easier to understand than the English code. Taking the definition of a most basic click operation as an example, the expected grammar rule is "当点击 [元素名称]", you can use the following definition:

   // Cucumber使用正则表达式匹配引号中的内容作为type参数
   @当("^点击 \"([^\"]*)\"$")
   public void findElementAndClick(String type) throws Throwable {
       // driver为Appium对待测设备的抽象，所有测试步骤最终转为对driver对操作
       // type可以传入元素ID对应的字符串，By.id表示通过元素resource-id查找
       driver.findElement(By.id(type)).click();
   }

When writing a case, use the commonly used Page Object design pattern for UI automation testing, that is, define a Page object for the UI page that needs to be tested in the APP, which contains the operable or verifiable elements on the page, and adds common methods.

Take the Huajiao homepage as an example, you can create a new object named "Homepage", which contains search methods corresponding to elements such as "Search", "My", and "Start Broadcast" (for example, search button, corresponding to the search element resource-id is com.huajiao:id/main_home_top_search). Since entering the user uid on the search page to search is a common operation, you can define a "search" method for this. All test cases, Page objects, elements, and methods are saved and edited using the test background webpage, and the basic keyword completion function is realized.

Define the basic operations such as clicking, sliding, and inputting text as above. After establishing the appropriate page and method, a use case can be transformed into a case description that is close to nature ( #beginning with a comment line):

# "$首页.搜索"表示使用"首页"Page中的"搜索"元素
当 点击 $首页.搜索
# "$搜索.搜索()"表示调用搜索页面的搜索方法，括号内为搜索关键词参数
$搜索.搜索(43011080)
当 断言元素出现 $搜索.搜索结果

Write code to perform complex custom operations

Using Cucumber to define common operations, such as clicking, sliding, verifying text, etc., can reduce the workload of writing a test case and improve the readability of the test case, but not all functions can use common operations. Especially because Cucumber only supports the orderly execution of instructions step by step, and cannot branch or loop instructions, so complex operation logic requires writing code in custom steps to complete the operation. The encapsulation of the code part refers to the Espresso project officially provided by Android, and the process of "find-operate-verify" is carried out by chain call.

Take the Android client logout as an example, click the "Home-My" element at the bottom. If the current status is not logged in, a login pop-up will pop up. At this time, the "Home-My" element at the bottom is not visible, indicating that you have not logged in. status.

Because Cucumber is executed sequentially and cannot be performed "我的"元素可见时退出登陆，不可见时关闭登陆弹窗, you need to write code to customize the logout steps:

    @当("^退出登录$")
    public void logout() throws Throwable {
        // 点击"首页-我的"
        onView(By.id("com.huajiao:id/bottom_tab_user")).perform(click());
        try {
            // 如果当前用户已登陆，不会弹窗提示登陆，"首页-我的"元素可见
            onView(By.id("com.huajiao:id/bottom_tab_user")).check(matches(isDisplayed()));
            // 调用退出登录的方法
            logOut();
        }
        // 未登录状态，"首页-我的"元素不存在，抛出NoSuchElementException
        catch (NoSuchElementException e) {
            // 点击系统back键关闭登陆弹窗
            onActions().pressKeyCode(AndroidKey.BACK, "1").perform();
        }
    }

Use Appium to find UI elements

Basic search method
- By.id: Search through the resource-id of the element;
- MobileBy.AndroidUIAutomator(String code): Search by code text of UIAutomator2. codeIn order to comply with the code text of the UIAutomator2 specification, Appium will parse the text and use reflection to call UIAutomator2 to search; the following is the use of the elements UiSelectorcontained textin the search text :String code = "new UiSelector().textContains(\"" + text + "\");";
xpath find elements
xpath can be used to find elements and attributes in XML documents. Appium and Google's official uiautomatorviewer tool to obtain elements are organized in xml format, and xpath can accurately locate elements By.idthat By.classNamecan only be located but cannot be located:
- The copy is a sibling element of the "TEXT" element, and the resource-id of the sibling element is "ID":
  xpath://*[@text='TEXT')]/../android.widget.TextView[@resource-id='ID']
- The resource-id is "ID" and the child element of the selected state element. The attr attribute of the child element is value: xpath://*[@resource-id='ID' and @selected='true']/*[@attr='value']
Although the xpath method is more accurate to find elements, the path of the elements may be affected by layout changes, and the performance on iOS is not good. Therefore, it is recommended to use resource-idmethods such as combining positioning elements first
Image recognition search element
Appium supports searching by picture at the By Selector level By by = MobileBy.image(base64ImageString). Currently, multi-element search is not supported, only the first found element is returned.
For Appium to support image search, a little preparatory work is required:
1. Install the NodeJS version of the OpenCV library:npm install -g opencv4nodejs
2. Configure related parameters in Appium:
```
// 设置图片识别阈值，默认0.4。需要尝试在找不到元素和找到不匹配元素间的平衡
driver.setSetting(Setting.IMAGE_MATCH_THRESHOLD, 0.5);
// 图片识别耗时较长，可以在操作元素对时候不再次查找图片，以节省时间
driver.setSetting(Setting.CHECK_IMAGE_ELEMENT_STALENESS, false);
```

StaleElementReferenceException: When Appium finds an element and then attempts to manipulate the element, StaleElementReferenceExceptionan exception will be thrown if the element is no longer on the DOM resource of the current page . When Appium uses UIAutomator2 to find elements, it will keep the cache of the elements. When operating on the elements, it will directly pass the cached information to UIAutomator2 for click, slide and other operations.

In the actual test process, there may be steps: page A jumps to page B; click element el on page B. Both pages A and B have elements with the same ID as el. When trying to manipulate element el on page B, Appium directly uses the cache of page A, which will appear at this time StaleElementReferenceException;
Since Appium uses HTTP requests to find and manipulate elements, the actual process of finding and operating elements is: POST finding element -> server cache element -> POST operating cache element, there is a time interval. It may also be caused if elements such as pop-up windows on the APP end are blocked during the network request StaleElementReferenceException.

Overall workflow

The htest client client obtains the download list of the packaged Android packaging server, and filters the latest APK installation package version from it. If there is a higher version than the latest version on the mobile phone, the Huajiao APP on the mobile phone will be overwritten and the BVT test case execution will be triggered automatically (triggered directly from the test platform web page when executing a single case);
The test platform selects the BVT use case set described by Cucumber, and at the same time finds the Page page, escapes the elements and methods of the use case step, and replaces it with the element locator that can be used by the client (the id:beginning means searching through resource-id, and the text:beginning means searching through text content ), which is returned to the client via an HTTP request (using socket to send when executing a single case).
During the execution of the test case, it may happen that the mobile phone pop-up window covers the Huajiao APP element when searching for the element. Therefore, during the execution of the test case, it will detect the pop-up window that may appear on the mobile phone and not expected in the test step. , Including the first charging popup, opening gift download popup, etc., after closing the popup, look for the element again;
htest client initializes Appium driver, uses Appium as a proxy to connect to the mobile phone, and performs basic operations in the test case on the mobile phone;
If the execution of the test case fails, it will try to re-execute the failed case. If it fails again, it will collect mobile phone logs, save screenshots and screen recordings, and save the failed log back to the test platform. Use socket to send execution results when executing a single case , The result is returned to the test platform through htest Server for display, if bvt, the result data is returned through the interface

Use the test platform to execute test cases at a time on the web:

If you exchange experience in software testing, interface testing, automated testing, and interviews. If you are interested, you can add software test communication: 1085991341, and there will be technical exchanges with colleagues.
Divided by modules, the entire framework is divided into:

Test platform: Web page, used to save and edit Cucumber-based test cases, manage Page pages, parse the elements in the use cases, and send the escaped use cases to the client to display the actual execution results of the client;

htest server: Java middleware, netty framework used, responsible for forwarding socket messages, that is, the test platform notifies the client to execute the use case message, and the client execution result returns to the test platform. use:

Start of netty on the server side in htestcom.htest.server.server.BaseServer

@Overridepublic void run() {
    if (bossGroup == null) {
        bossGroup = new NioEventLoopGroup();
        model.setBossGroup(bossGroup);
    }
    if (workerGroup == null) {
        workerGroup = new NioEventLoopGroup();
        model.setWorkGroup(workerGroup);
    }
    ServerBootstrap b = new ServerBootstrap(); 
    b.group(model.getBossGroup(),model.getWorkGroup())
        .channel(NioServerSocketChannel.class)
        .option(ChannelOption.SO_BACKLOG, 100)
        .option(ChannelOption.SO_KEEPALIVE, true)
        .handler(new LoggingHandler(LogLevel.INFO))
        .childHandler(getChildHandler());
    try {
        future = b.bind(SERVER_IP, getPort()).sync(); 
        LOGGER.debug("服务启动成功 ip={},port={}",SERVER_IP, getPort());
        future.channel().closeFuture().sync();
    } catch (Exception e) {
        LOGGER.error("Exception{}", e);
    } finally {
        Runtime.getRuntime().addShutdownHook(new Thread() {
            @Override public void run() {
                shutdown();
            }
        });
    }
}

HttpServer, JarServer, and WebsocketServer are all started in the same way. The difference is that they listen on different ports and handle different data handlers.

The processor of HttpServer is com.htest.server.handler.ServerHttpHandlerthat processing messages are processed in accordance with the http protocol

@Override
protected void messageReceived(ChannelHandlerContext ctx, HttpRequest request) {
    try {
        this.request = request; headers = request.headers();
        if (request.method() == HttpMethod.GET) {
            QueryStringDecoder queryDecoder = new 
                QueryStringDecoder(request.uri(), Charset.forName("utf-8")); 
            Map<String, List<String>> uriAttributes = queryDecoder.parameters(); //此处仅打印请求参数（你可以根据业务需求自定义处理） 
            for (Map.Entry<String, List<String>> attr : uriAttributes.entrySet()){
                for (String attrVal : attr.getValue()) {
                    Logs.HTTP.debug(attr.getKey() + "=" + attrVal);
                }
            }
        }
        if (request.method() == HttpMethod.POST) {
            fullRequest = (FullHttpRequest) request;
            //根据不同的 Content_Type 处理 body 数据
            dealWithContentType();
        }
        keepAlive = HttpHeaderUtil.isKeepAlive(request);
        writeResponse(ctx.channel(), HttpResponseStatus.OK, "开始执行", keepAlive);
    } catch (Exception e) {
        writeResponse(ctx.channel(), HttpResponseStatus.INTERNAL_SERVER_ERROR, "启动失败", true);
    }
}

JarServer's processor is com.htest.server.handler.ServerHandlerthat processing messages are processed in accordance with the protobuf format

@Override
protected void handleData(ChannelHandlerContext ctx, MessageModel.Message msg) {
    Connection connection = server.getConnectionManager().get(ctx.channel());
    connection.updateLastReadTime();
    server.getMessageReceiver().onReceive(msg, connection);
}

The processor of WebsocketServer is com.htest.server.handler.ServerChannelHandlerthat it also processes messages according to the protobuf format. The difference from HttpServer is that their ChannelInitializer is different

htest client: Java client, used to define Cucumber steps, update the phone APK, initialize Appium, and execute test cases; Usage: execute in the command line on the PC side java -jar htest-client.jar, the PC side needs to have Appium and nodejs opencv environment, controlled by the yaml configuration file End parameters during the execution of the test. The specific working methods are as follows:
- Function: The jar supports regular checking of the latest apk function, which is disabled by default, and is enabled through the yaml file configuration. If the latest apk is found, it will be automatically installed on the mobile phone, and a request will be sent to the web server (the test platform for automated case management) to trigger the execution of the specified module case set.
- Download strategy: The system downloads only the latest apk by default, if the apkVersion value in the local yaml configuration file is higher than the apkVersion value on the server. If it is smaller than the server, it will not be downloaded.
- Installation strategy: After the download is complete, the versionName of the apk in the phone (parsed by aapt) will be compared with the versionName of the downloaded apk. If the downloaded apk is new, install it, otherwise it will not be installed. The configuration parameters can also be installed to the designated mobile phone. If there is only one mobile phone, no configuration parameters are required.
- After the installation is complete, the apkVersion value will be automatically updated for the next judgment.
- After the installation is complete, it will send an http request to the web server. After the web server receives it, it will trigger it once and send it to the current mobile phone case set task. The specific case set module is configured by the models parameter, and the mail recipient is configured through mails.
Appium: NodeJS client/server, used to connect to the mobile phone, through UIAutomator2/XCUITest, perform basic operations such as getting elements/clicking/swiping on the mobile terminal; the
above content is the entire content of this article . The above content is hoped to be helpful to you. Friends who helped are welcome to like and comment.