Mobile phone group control system OpenSTF

Before doing private domain traffic, I studied a lot of WeChat group messages, and accidentally discovered stf, an excellent open source mobile phone group control system, which can realize remote and real-time control of mobile phones. An old Android machine and a Raspberry Pi can You can build a system that can realize cloud mobile phone for yourself in real time.

And it can be customized and developed for the mobile phone wall, point wall, automated testing, and batch execution for brushing the list, which is extremely simple.

I searched for some information on the Internet, but it was all copy and pasted, and there was no nutrition. All are simple to build and easy to use. This time I would like to take this opportunity to sort out all the pits I stepped on and make detailed records.

Let's take a closer look at OpenSTF, and sort it out from several angles

1. Build the OpenSTF system

2. Connect mobile devices

3. System role permissions

4. Remote debugging

5. Remote Access Token

6. Jenkins integration

7. App control

8. Distributed deployment of OpenSTF

(1) Building the OpenSTF system

There are two ways:

1. Through Docker

2. Build it yourself according to the steps (there are many pits, not recommended)

In view of the versatility of system construction, we only talk about the situation of building through docker for the time being.

Docker needs three images to build stf: stf, rethinkdb, adbd

It is recommended to use docker-compose to configure:

version: "3"
services:
  rethinkdb:
    image: rethinkdb
    container_name: rethinkdb
    restart: always
    volumes:
      - "/docker/openstf/rethinkdb:/data"

  adbd:
    image: sorccu/adb:latest
    container_name: adbd
    privileged: true
    restart: always
    volumes:
      - "/dev/bus/usb:/dev/bus/usb"
      - "/data:/data"

  stf:
    build: .
    container_name: stf
    restart: always

After configuration, execute docker-compose up directly. Configuration via docker is very simple.

After the image is pulled and started, you will see three containers in docker ps: openstf/stf:latest, rethinkdb, sorccu/adb:latest.

Notice! ! ! There is a hole here!

Since stf needs a large number (thousands) of ports to connect to the mobile phone, it is necessary to open all the firewall ports of the server, otherwise it will not be able to connect to the mobile phone! !

(2) Connecting mobile devices

Originally, the step of connecting the mobile device can be directly plugged in, but there will be some problems in actual operation. This section only summarizes the problems:

1. You need to turn on the phone's USB debugging

2. You need to open the phone to allow simulated location clicks (in developer mode)

3. The STF Service APP will be installed automatically (some mobile phones need to click to allow)

After completing the above three items, you can see the corresponding mobile device from the background of stf and operate it.

It should be noted that the mobile phone connected to STF needs to have a minicap, otherwise it cannot be connected normally. It is currently known that some high-version OPPO/VIVO mobile phones cannot be connected. If you want to do group control, it is recommended to use a model around Mi 6. 

(3) System role permissions

OpenSTF itself supports a variety of authentication methods, this section provides a more common way LDAP. It can be accessed seamlessly, and permissions can be configured through LDAP

./stf local --public-ip ${ip} --allow-remote --auth-type ldap --auth-options ‘["–ldap-url",“ldap:服务器地址”, “–ldap-bind-dn”,“你的账号”,"–ldap-search-dn",“DC=sunraycn,DC=cn”, “–ldap-bind-credentials”,“ldap密码”, “–ldap-search-class”,“user”, “–ldap-search-field”, “用户信息字段”]’

 (4) Remote debugging

First you need to register a keychan in STF

 Generate a keychan on the debugging computer that you need to connect to the remote mobile phone, then paste it and save it. Fill in a memorable name for the device.

After adding keychan, we can choose a mobile phone to control. After entering, we can see the following screen:

Copy the remote connection command in the red box and execute it in your bash to remotely connect to the remote device through adb.

Remember to enter when you end debugging

adb disconnect

to disconnect the remote connection.

Notice! ! ! Only when the mobile phone control is turned on in the background page, remote debugging can be performed, otherwise the port will not be opened and debugging cannot be performed! ! ! 

(5) Remote access token

STF can access the API of the system through the remote access token

Generate a token in the STF background:

Be sure to save it after generation and use it in future API requests.

Commonly used APIs of STF:

1. Get user information
Method: GET
URL: http://yourdomain/api/v1/user
Header:
Authorization: Bearer 2a91d54cceb54993a468582041c8c67cadb6116881f84fec825e5111bdcde6c9

2. List all STF devices (including disconnected or otherwise inaccessible devices)
Method: GET
URL: http://yourdomain/api/v1/devices
Header: Authorization: Bearer 2a91d54cceb54993a468582041c8c67cadb6116881f84fec825e5111bdcde6c9

3. Return information about a specific device
Method: GET
URL: http://yourdomain/api/v1/devices/FA6990313540
Header: Authorization: Bearer 2a91d54cceb54993a468582041c8c67cadb6116881f84fec825e5111bdcde6c9

4. Return information about yourself (authenticated user)
Method: GET
URL: http://yourdomain/api/v1/user
Header: Authorization: Bearer 2a91d54cceb54993a468582041c8c67cadb6116881f84fec825e5111bdcde6c9

5. Return a list of devices currently being used by the authenticated user
Method: GET
URL: http://yourdomain/api/v1/user/devices
Header: Authorization: Bearer 2a91d54cceb54993a468582041c8c67cadb6116881f84fec825e5111bdcde6c9

6. Attempt to add a device under the control of an authenticated user. This is similar to pressing "use" in the UI
Method: POST
URL: http://yourdomain/api/v1/user/devices
Header:
(1) Authorization: Bearer 2a91d54cceb54993a468582041c8c67cadb6116881f84fec825e5111bdcde6c9
(2) Content-Type: application/json
Body: {"serial":"521df70f"}

7. Remove the device from the authenticated user's device list. This is similar to pressing "Stop using" in the UI
Method: DELETE
URL: http://yourdomain/api/v1/user/devices/521df70f
Header: Authorization: Bearer 2a91d54cceb54993a468582041c8c67cadb6116881f84fec825e5111bdcde6c9

8. Obtain device remote connection url
Method: POST
URL: http://yourdomain/api/v1/user/devices/521df70f/remoteConnect
Header: Authorization: Bearer 2a91d54cceb54993a468582041c8c67cadb6116881f84fec825e5111bdcde6c9

9. Remote debugging session of the disconnected device
: delete
url: http:// yourDomain/API/V1/User/DEVIES/521DF70F/RemoteConnect
Header: Authorization: Bearer 2A91D54CCEB549999999 3A468582041C8C67CADB6116881F84FEC825E5111BDCDE6C9
 

(6) Jenkins integration

There is an OpenSTF plug-in in Jenkins that can be installed, but the version is relatively old and no one seems to be maintaining it.

After installation, in the build step, you can add an idle mobile phone in STF to execute the build content

(7) App control

Since we recommend using Docker to build OpenSTF, you can use adb when you want to control and perform some automated operations on the app. Different from adb debug, you can directly invoke adb commands by entering the docker container. The following is an example of opening WeChat:

docker exec openstf_adb_1 adb -s $element shell am start com.tencent.mm/com.tencent.mm.ui.LauncherUI

Because multiple devices are connected in STF, it is necessary to use adb -s to distinguish which device is connected. Of course, if you use jenkins to build, you can directly write a shell to traverse the device id.

Similarly, monkey testing and UI automation testing can be performed through the command line.

(8) Distributed deployment of STF

There are two schemes for STF distribution, the schemes are as follows:

• The master node deploys the STF service and allows remote access to the API, and the child nodes only need to deploy ADB to ensure that the ADB 5037 port is exposed to the outside world. The advantage of this solution is that it is easy to deploy, and the child nodes do not need to deploy STF services. The disadvantage is that the new machine on the child node needs to re-run the STF command on the master node to identify the new machine on the child node.
• The master node deploys the STF service and allows remote access to the API, and the sub-node also deploys the STF service and allows remote access to the API. The master node then connects the child nodes. The advantage of this solution is that the newly added device of the child node can be automatically identified. The disadvantage is that child nodes also need to deploy STF services.

In the first case, take the Window child node as an example

(1) Open ADB and expose port 5037 to the outside world

adb nodaemon server -a -P 5037

If the adb port is occupied, you can use the command to find the process occupying port 5037 and kill the process.

netstat -aon|findstr 5037

taskkill /pid 5018 /f

(2) Go to the MAC Master master node and run the stf provider command to connect to the Window child node

Window IP: 192.168.27.247, MAC machine IP: 192.168.27.180

stf provider --name GHL.local --min-port 7400 --max-port 7700 --connect-sub tcp://127.0.0.1:7114 \
--connect-push tcp://127.0.0.1:7116 --group-timeout 900 --public-ip 192.168.27.180 \
--storage-url http://localhost:7100/ \
--adb-host 192.168.27.247 --adb-port 5037 \
--vnc-initial-size 600x800 --mute-master never --allow-remote

For so many parameters above, you can check the master node STF startup log to find out, here is mainly by adding

--adb-host 192.168.27.247 --adb-port 5037 \

to connect to the Windows machine.

In the second case, take the Linux child node as an example

Deploy an STF service on Linux and run the remote access API. The IP of Linux is: 192.168.27.200, and the IP of the MAC machine is: 192.168.27.180. Then we only need to run stf provider on Linux to connect to the Master node. The command is as follows:

stf provider --name centerOs.local --min-port 7400 --max-port 7700 \ 
--connect-sub tcp://192.168.27.180:7114 --connect-push tcp://192.168.27.180:7116 \
--group-timeout 20000 --public-ip 192.168.27.180 --storage-url http://192.168.27.180:7100/ \ 
--vnc-initial-size 600x800 --allow-remote

Here you can see that there is no need to enter the adb remote port. At this time, the machine is inserted into the Linux node, and the Master node can view the machine mounted on Linux in real time. There is no need to re-enter the stf provider command on the Master node.

To sum up, OpenSTF is still a relatively general and simple open source group control integration tool.

The landlord guessed that the cloud test systems of major platforms are secondary developed based on OpenSTF. Of course, the cloud test systems of major platforms will have more complex task distribution, authority control, log recovery, and report output.