WeChat applet cloud testing service provides a wealth of automated testing capabilities for users to choose from, among which the simplest and most convenient one is the intelligent Monkey service. It can be used after the user accesses it. It is very suitable for small programs with few pages and simple functions, and can realize quick smoke testing with zero code and low cost.
Intelligent Monkey
The overall test process of Intelligent Monkey is to automatically establish an exploration path through intelligent clicking during the test, and realize the intelligent traversal of the applet page. During the traversal process, it will automatically obtain performance data , detect abnormal conditions , such as whether JsError occurs, black and white screen, etc., and finally generate a complete test report.
At present, for the intelligent Monkey test, whether there is an abnormality is the criterion for judging whether the test is passed. In the future, the test dimension of the Monkey service will be optimized and the real machine experience scoring data will be integrated. Stay tuned~
The figure below shows a sample of the Monkey test report of a certain project. It can be seen that some JsError errors in the experience version of the applet were found in this test. At this time, the test students can notify the development to fix it in time
Smart click strategy
Why do we call it Intelligent Monkey here, and what is the difference between it and ordinary Monkey?
The usual Monkey test is to click on as many elements as possible by random clicking, and test whether there will be a crash or the like.
Through the introduction of Smart Monkey, it can be found that the purpose of the Mini Program Monkey test is to do a smoke test in the page dimension , and find out whether there are abnormal problems such as JsError and black and white screens on the page, so the goal of the Mini Program Smart Monkey is to cover as much as possible page, instead of clicking all elements of the page once
Generally speaking, the Monkey process is:
- Element analysis, identifying which elements can be clicked on the page,
- Select an element click
- Backtracking when you can’t click.
The cloud measurement service has customized and optimized the above steps according to the characteristics of the small program, and also added historical experience accumulation and distributed strategies to improve coverage.
element analysis
In terms of element analysis, we started to use machine learning to mark more than 1,000 small programs, and trained a set of models through yolov3 to identify clickable elements. Recently, we have made improvements to obtain the DOM tree of the current page of the applet through JS injection, and identify elements by parsing the DOM tree. From the figure, we can see the JS injection method, which can more accurately identify the clickable elements of the applet, which is closer to human perception
Select the clicked element
In terms of selecting clickable elements, we filter and filter according to the rendered attribute characteristics of the elements in the DOM tree of the page. For example, select elements that are bound to click or slide events first, and filter elements that are not in the current visible area or invisible.
page backtracking
When the click cannot go on and the page needs to be backtracked, you can directly return to the page in a certain state by spelling path+query according to the characteristics of the applet
Accumulation of historical experience
After each test, the background will record the success of the page jump in each test, draw it into a directed graph, and use it as historical experience to select elements for subsequent tests as a reference. So in general, the more times of running tests, the higher the coverage.
distributed strategy
When the user submits multiple machines for simultaneous testing, for example, when submitting 10 Android machines for simultaneous testing, the background will try to let different machines cover and click on different pages, so as to improve the overall task success rate
current effect
Judging from the current statistical results, the current 20-minute average page coverage for both Android and iOS is around **30%**.
The follow-up cloud test service will further optimize the algorithm and combine the current web page access situation. For example, if there are 100 pages in the applet, there may be only 10-20 pages that are accessed more on the live network, so in the limited test time, try to click on it first. Live network users visit many pages , which is more in line with the actual needs of the business
Custom Monkey
Some users want to further improve the coverage of the Monkey page . At this time, you can try to customize the Monkey
Before starting the custom Monkey test, the user needs to go to the test plan page to configure the specified page list, and open the specified page list directly through the configured path+query method during the test.
After the opening is completed, the cloud test service provides a variety of successor modes for subsequent intelligent testing:
- Free exploration: After opening all specified pages, continue the original intelligent exploration to further improve coverage
- Key coverage: After all specified pages are opened directly, return to the key page and continue clicking. This mode is very suitable for some projects who want to do page stability testing , and hope to repeatedly click on several key pages to test page stability
- End test: directly end the test task and quickly obtain test results
Quick Monkey
In addition to ordinary mini program developers, there are also third-party mini program service providers who help authorized merchants develop mini programs on their behalf, such as Youzan, Weimob, Mingyuan Cloud, etc.
For users of third-party service providers, there are many small programs authorized for them to develop. At this time, Quick Monkey is very suitable for them to do a quick smoke test of multiple small programs.
It supports a test of up to 30 small programs at a time, and each small program runs for 1 minute. Intelligent Monkey , checks for abnormal conditions, and also supports smoke tests for specified pages for each small program, just like custom Monkey
Please note that only third-party service providers can use the Quick Monkey mode!
Summarize
The three Monkey modes introduced above can help users to conduct quick smoke tests, and they are all zero-code, and can be used by connecting to the cloud test service.
But intelligent Monkey also has some disadvantages, such as:
- Non-click operations such as input and image upload are not supported
- Do not verify page logic verification, such as asserting whether an element exists, etc.
Then how to solve these problems, please refer to "WeChat Mini Program Automation Testing" follow-up series of articles
WeChat Mini Program Automation Testing - Recording and Playback
WeChat Mini Program Automation Testing - Custom Testing