The strongest in the whole network, Python+Appium+pytest automated testing, multi-device concurrency + multi-threading (actual details)

Contents: Guide

foreword

Appium+python implements single-device app automation testing

Start the appium server, occupying port 4723;
the computer is connected to a device, and the connected device is obtained through adb devices;

In the python code, write startup parameters and write test cases through pytest for automated testing.

To execute automated tests concurrently on multiple devices

Need:
Determine the number of devices;
each device corresponds to a port number of appium server, and start appium;
pytest needs to obtain the startup parameters of each device, and then execute the automated test. ;

implementation strategy

Step 1: Obtain the currently connected device from the device pool. If the device pool is empty, no device is connected.

Step 2: If the device pool is not empty, start a thread to start the appium server. Corresponding to the number of devices.
The initial server port is 4723. For each additional device, the port number defaults to +4

Step 3: If the device pool is not empty, enable multiple threads to execute app automation testing.

Concrete implementation steps

1. Use the adb command to obtain the number of currently connected devices, device name, and Android version number of the device.
Define a ManageDevices class.

Restart the adb service.
Use the adb devices command to obtain the number of connected devices in the current platform, and the device uuid.
Use adb -P 5037 -s device uuid shell getprop ro.build.version.release to obtain the version number of each device.
Store the device names and device version numbers of all connected devices in a list.
The list in 4 can be obtained by calling the get_devices_info function.

Part of the implemented code is:

class ManageDevices:
    """
       1、重启adb服务。
       2、通过adb devices命令获取当前平台中,已连接的设备个数，和设备uuid.
       3、通过adb -P 5037 -s 设备uuid shell getprop ro.build.version.release获取每一个设备的版本号。
       4、将所有已连接设备的设备名称、设备版本号存储在一个列表当中。
       5、通过调用get_devices_info函数，即可获得4中的列表。
    """

    def __init__(self):
        self.__devices_info = []
        # 重启adb服务
        self.__run_command_and_get_stout("adb kill-server")
        self.__run_command_and_get_stout("adb start-server")

    def get_devices_info(self):
        """
        获取已连接设备的uuid,和版本号。
        :return: 所有已连接设备的uuid,和版本号。
        """
        self.__get_devices_uuid()
        print(self.__devices_info)
        self.__get_device_platform_vesion()
        return self.__devices_info

2. Define a device configuration pool.

Device startup parameter management pool.
Each device: corresponds to a startup parameter and the port number of the appium service.

The startup parameter template is stored in the desired_caps_config/desired_caps.yaml file.
Read the startup parameters from the template in 1.
From the device list, obtain the device uuid and version number of each device, and merge them with the startup parameters in 2.
For each device, specify an appium service port number. Starting from 4723, each additional device will increase by 4 by default

For each device, specify a port number for local tcp communication with the device. Starting from 8200, each additional device increases by 4 by default.

Among the startup parameters, it is specified by systemPort.
Because the appium service will specify a local port number to forward the data to the Android device.
By default, port 8200 is used. When there are multiple appium services, port conflicts will occur. It will cause socket hang up error during operation.

Part of the code implemented:

def devices_pool(port=4723,system_port=8200):
    """
    设备启动参数管理池。含启动参数和对应的端口号
    :param port: appium服务的端口号。每一个设备对应一个。
    :param system_port: appium服务指定的本地端口，用来转发数据给安卓设备。每一个设备对应一个。
    :return: 所有已连接设备的启动参数和appium端口号。
    """
    desired_template = __get_yaml_data()
    devs_pool = []
    # 获取当前连接的所有设备信息
    m = ManageDevices()
    all_devices_info = m.get_devices_info()
    # 补充每一个设备的启动信息，以及配置对应的appium server端口号
    if all_devices_info:
        for dev_info in all_devices_info:
            dev_info.update(desired_template)
            dev_info["systemPort"] = system_port
            new_dict = {
    
    
                "caps": dev_info,
                "port": port
            }
            devs_pool.append(new_dict)
            port += 4
            system_port += 4
    return devs_pool

Special note: When 2 or 2 devices are used concurrently, an error message of device socket hang up will be encountered.

what is the reason?
In the log of appium server, there is such a line of adb command: adb -P 5037 -s 08e7c5997d2a forward tcp:8200 tcp:6790

What does that mean?
Forward the data of the local port 8200 to the port 6790 of the Android device.
Therefore, if multiple appium servers are started locally, they all use port 8200, and conflicts will occur.

Solution:
It should be set that each appium server uses a different local port number to forward data to different devices.
Among the startup parameters: add systemPort= port number to set.
This way, each device uses a different local port, and the problem is resolved.

3. Start and stop management of appium server

ps: Here you can use the appium command line version or the desktop version

The appium server must be started before the automation use case can run.
After the automation use case is executed, kill the appium service. This will not affect the next run.

The code is implemented as follows:

import subprocess
import os

from Common.handle_path import appium_logs_dir

class ManageAppiumServer:
    """
    appium desktop通过命令行启动appium服务。
    不同平台上安装的appium，默认的appium服务路径不一样。
    初始化时，设置appium服务启动路径
    再根据给定的端口号启动appium
    """

    def __init__(self,appium_server_apth):
        self.server_apth = appium_server_apth

    # 启动appium server服务
    def start_appium_server(self,port=4723):
        appium_log_path = os.path.join(appium_logs_dir,"appium_server_{0}.log".format(port))
        command = "node {0} -p {1} -g {2} " \
                  "--session-override " \
                  "--local-timezone " \
                  "--log-timestamp & ".format(self.server_apth, port, appium_log_path)
        subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE,shell=True).communicate()

    # 关闭appium服务
    @classmethod
    def stop_appium(cls,pc,post_num=4723):
        '''关闭appium服务'''
        if pc.upper() == 'WIN':
            p = os.popen(f'netstat  -aon|findstr {
      
      post_num}')
            p0 = p.read().strip()
            if p0 != '' and 'LISTENING' in p0:
                p1 = int(p0.split('LISTENING')[1].strip()[0:4])  # 获取进程号
                os.popen(f'taskkill /F /PID {
      
      p1}')  # 结束进程
                print('appium server已结束')
        elif pc.upper() == 'MAC':
            p = os.popen(f'lsof -i tcp:{
      
      post_num}')
            p0 = p.read()
            if p0.strip() != '':
                p1 = int(p0.split('\n')[1].split()[1])  # 获取进程号
                os.popen(f'kill {
      
      p1}')  # 结束进程
                print('appium server已结束')

4. Execute automated test cases according to different startup parameters in pytest

When using pytest to execute use cases, pytest.main() will automatically collect all use cases and automatically execute the generated results.

In this case, the startup information of the appium session is given in the code.

In the above mode, only the startup information of one device will be read and a session with the device will be started.

Although the fixture has parameters to pass multiple device startup information, it is executed serially.

The problem that needs to be solved is:
you can pass the startup parameters of multiple devices, but not the parameters of the fixture.
Every time a device startup parameter is passed in, execute pytest.main() once

Solution:
via pytest's command line arguments. That is, among the parameters of pytest.main(), the startup information of the device is passed in.
Use python's multithreading to achieve. Every time a device startup parameter is received, a thread is started to execute pytest.main

1) The first one is the command line parameter of pytest.
First, you need to add command-line options in conftest.py, and pass in the parameter "-cmdopt" from the command line.

If the use case needs to use the parameters passed in from the command line, the cmdopt function is called.

code show as below:

def pytest_addoption(parser):
    parser.addoption(
        "--cmdopt", action="store", default="{platformName:'Android',platformVersion:'5.1.1'}",
        help="my devices info"
    )


@pytest.fixture(scope="session")
def cmdopt(request):
    return request.config.getoption("--cmdopt")


@pytest.fixture
def start_app(cmdopt):
    device = eval(cmdopt)
    print("开始与设备 {} 进行会话，并执行测试用例 ！！".format(device["caps"]["deviceName"]))
    driver = start_appium_session(device)
    yield driver
    driver.close_app()
    driver.quit()

2) Use multi-threaded implementation: every time a device startup parameter is received, a thread is started to execute pytest.main

Define a main.py.

run_case function.
This method is mainly: receiving device startup parameters, and collecting and executing use cases through pytest.main.

# 根据设备启动信息，通过pytest.main来收集并执行用例。
def run_cases(device):
  """
  参数：device为设备启动参数。在pytest.main当中，传递给--cmdopt选项。
  """
    print(["-s", "-v", "--cmdopt={}".format(device)])
    reports_path = os.path.join(reports_dir,"test_result_{}_{}.html".format(device["caps"]["deviceName"], device["port"]))
    pytest.main(["-s", "-v",
                 "--cmdopt={}".format(device),
                 "--html={}".format(reports_path)]
                )

Every time there is a device, start a thread and execute the run_cases method.

# 第一步：从设备池当中，获取当前连接的设备。若设备池为空，则无设备连接。
devices = devices_pool()

# 第二步：若设备池不为空，启动appium server.与设备个数对应。起始server端口为4723，每多一个设备，端口号默认+4
if devices and platform_name and appium_server_path:
    # 创建线程池
    T = ThreadPoolExecutor()
    # 实例化appium服务管理类。
    mas = ManageAppiumServer(appium_server_path)
    for device in devices:
        # kill 端口，以免占用
        mas.stop_appium(platform_name,device["port"])
        # 启动appium server
        task = T.submit(mas.start_appium_server,device["port"])
        time.sleep(1)

    # 第三步：若设备池不为空，在appium server启动的情况下，执行app自动化测试。
    time.sleep(15)
    obj_list = []
    for device in devices:
        index = devices.index(device)
        task = T.submit(run_cases,device)
        obj_list.append(task)
        time.sleep(1)

    # 等待自动化任务执行完成
    for future in as_completed(obj_list):
        data = future.result()
        print(f"sub_thread: {
      
      data}")

    # kill 掉appium server服务,释放端口。
    for device in devices:
        ManageAppiumServer.stop_appium(platform_name, device["port"])

The following is the most complete software test engineer learning knowledge architecture system diagram in 2023 that I compiled