Table of contents
01. The principle of distributed testing
Automated testing learning resource sharing
I don’t know if you have encountered such a situation. In actual work, there may be hundreds of test cases for app automation testing. If you run it on one machine, it will take a long time. Can you use distribution How to run test cases? For example, if there are 1,000 test cases, assign 500 to machine A and 500 to machine B, and run them at the same time, which will greatly reduce the time-consuming. Just pytest-xdist provides us with a possibility
What is pytest-xdist?
pytest-xdist is a distributed testing plugin. It has made some unique test execution mode extensions based on pytest. For example, if you have multiple CPUs or multiple machines, you can use them to do some parallel tests, and it is also cross-platform, you can specify different python interpreters or different platforms to execute the test
Use the pip command to install pytest-xdist easily and quickly
pip install pytest-xdist
01. The principle of distributed testing
As mentioned above, there are two ways for pytest-xdist to implement distribution: one is to use multiple CPUs, and the other is to use multiple machines.
Generally speaking, for web automated testing, it is more appropriate to use multi-core CPUs for distributed testing, but for app automated testing, it is more appropriate to use multiple machines for distributed testing. The latter is mainly introduced here
The way pytest-xdist works is that one master corresponds to multiple workers, and each worker executes its own test set according to the requirements of the master.
The basic execution flow is:
-
The master spawns one or more workers at the beginning of a test session. The master and worker use execnet and its gateway to communicate. Workers can be local or remote
-
The worker acts as the executor of pytest. After collecting a complete test set, the worker sends the collected test id to the master that does not undertake the execution task
-
The master receives the results sent by all nodes, and also does some availability checks to ensure that all workers have collected the same test set. If all goes well, it converts the list of test ids into a list of simple indices, each index corresponding to its position in the original test set list. The feasibility of this scheme is that it can save bandwidth. Since all workers have the same test set list, the master can assign one of the workers to only execute the use case whose test index is 3.
pytest-xdist has two modes, each mode and load mode
-
each mode:
The master will send all test tasks (the test tasks here are not test sets, similar to test instructions, that is, the test index mentioned above) to each worker, and each worker executes a complete test set.
For example, if there are 1000 use cases divided into two workers A and B, then A and B execute 1000 use cases at the same time
-
load mode:
The master will first send 25% of the test tasks to each worker in a polling manner, and then distribute the rest after the worker finishes executing. Similarly, if there are 1,000 use cases and two workers, A and B, first distribute 250 use cases to A, then distribute 250 use cases to B, and then distribute the remaining 500 use cases after A and B are executed. Maybe the pressure on server A is relatively small at this time, then assign 260 entries to A, and the pressure on server B is relatively high, and assign 240 entries to B. Similar to the concept of load balancing
02. Test items
Taking the Baidu Tieba app as an example, you can write 4 simple use cases to test. The first test is the sliding function of the welcome page. If the user clicks Agree, slides the screen later, and can slide normally, click to experience immediately, and can jump normally, indicating that the use case is passed
The second test is that the welcome page clicks to disagree, and a prompt can pop up normally, indicating that the use case is passed
The third test search, if the user swipe the screen, enter keywords in the search bar on the homepage, and the specified content can be searched, indicating that the use case has passed
The last test login, click "My" on the homepage, you can enter the Baidu account login page, indicating that the use case has passed
03. Environment preparation
In the construction environment, I have prepared three virtual machines, one of which is used as the master, and the remaining two are used as workers for execution. In order to avoid unnecessary troubles, all three machines use python3.8 uniformly, both workers use docker to install appium1.17.0, and the client uses two night simulators with v7.1.2 system.
The specific information is shown in the table below:
server environment
| Role |
system |
python version |
ip |
| master |
Centos7.6 |
v3.8.0 |
192.168.0.109 |
| worker1 |
Centos7.6 |
v3.8.0 |
192.168.0.126 |
| worker2 |
Centos7.6 |
v3.8.0 |
192.168.0.136 |
client environment
| testing machine |
system |
ip |
tcpip |
| Night Simulator-1 |
v7.1.2 |
192.168.0.114 |
6666 |
| Night Simulator-2 |
v7.1.2 |
192.168.0.106 |
6666 |
operating environment
| container |
Version |
The port number |
host machine |
| app_1 |
v1.17.0 |
4725->4723 |
worker1 |
| appium_2 |
v1.17.0 |
4725->4723 |
worker2 |
04. Construction steps
1. Create and start the docker container on the worker
Perform the same operation on two workers to start a container named appium
Seeing this, you may ask how to start the appium service in docker. It is relatively simple. The code can start the appium service through the command line.
2. Simulator settings
By bridging, set the tcpip port so that the remote server can connect to the emulator through adb. The specific method is to click on the settings in the upper right corner, enter the system settings, property settings, and check to enable the network bridge mode. At this time, a bridge network card should be installed, and then you can set the static IP by yourself. Save and reboot
Under normal circumstances, the ip should become a static ip after restarting, but there is a bug in the Yeshen emulator, the ip is still a random ip, but this does not affect subsequent operations
Next, set the tcpip port. After setting it through the following command, if you see restarting in TCP mode port:6666, it means that the setting has been completed
3. appium remote connection simulator
Execute the remote connection command on the two workers respectively, worker1 corresponds to simulator 1, and worker2 corresponds to simulator 2. Here the main adb connect is followed by the actual ip address of the emulator plus the tcpip port number 6666. If you see connected to ip: port number, it means that the remote connection has been successful
4. Upload code
Use commands or tools to upload the code to the /opt directory on the master and decompress it
main.py is the entry point for project test execution. In order to adapt to distributed execution, we need to make some modifications to main.py
A description of the most critical steps:
-
"-d"：
Represents the distributed parameter
-
"--tx" followed by the address of the worker server
And the python version is specified by python=/opt/Python-3.8.0/bin/python3.8, and the directory of the synchronization test set on the worker is specified by chdir=/opt/pyexecnetcache, that is, the master will synchronize the test project with the worker under this directory
-
"--rsyncdir" followed by "./",
Indicates that all files in the current directory on the master will be synchronized
-
"TieBa-AutoTest"：
Because the synchronized project in the past is in the pyexecnetcache directory, which is actually /opt/pyexecnetcache/TieBa-AutoTest, if this configuration is not added, it will not be able to enter the TieBa-AutoTest directory, because pytest on the worker starts with test_ when executing The test case, so the test case will not be found, it can be understood as cd TieBa-AutoTest
05. Distributed execution
Execute the python3 main.py command on the master to perform distributed operation, gw0 and gw1 represent worker1 and worker2 respectively, and [4] represents a total of 4 test cases
At this point, go to the /opt directory of the worker , and the test project has been synchronized
Go to observe the two simulators again and find that they have started synchronous execution
06. Test report
After the test is completed, observe the execution result information on the master . The 4 test cases took less than 3 minutes, 2 passed and 2 failed.
But after generating the preview report through allure serve on the master, the report is found to be empty
The report previews on the two workers are normal, showing the report information of their respective execution cases
The reason is that pytest-xdist is not friendly to the allure test report support, and the test report information of each worker is not returned. Using pytest-html does not have this problem. Someone raised this issue on github. Leave a "suspense" for the time being, and talk about how to solve this problem next time
Automated testing learning resource sharing
Finally, in order to facilitate everyone to improve their testing skills, a 13G super-practical dry goods learning resource is specially prepared for you, and the content involved is very comprehensive.
These materials should be the most comprehensive and complete preparation warehouse for friends who want to advance [automated testing] . This warehouse has also accompanied me through the most difficult journey, and I hope it can help you too! Everything should be done as early as possible, especially in the technical industry, we must improve our technical skills.
