foreword
At present, more and more mobile devices are appearing in our daily life. People can efficiently complete many things through mobile devices, such as online shopping, recharge and payment, news information, financial management, etc. With the development of China's IT and Internet and the continuous maturity of the market, people's expectations for software functions are gradually increasing. Just imagine, when it takes 2 minutes for you to open an APP, the game freezes constantly, and the software power consumption is extremely fast during video communication, how distressing it will be, and the direct consequence is to give up using these applications. For the manufacturers of these applications, what they lose are users, markets and word of mouth, and these often determine the survival and development of enterprises.
There is no doubt that performance testing is a very important link in any software testing process, which directly affects user experience and user satisfaction.
1. APP performance test
The key indicators of APP performance testing mainly include: startup time, memory, CPU, power consumption, traffic, fluency, etc.
The basic principle of APP performance testing is to identify and discover code quality problems in APP by monitoring various indicators of APP in various test scenarios, and to correct and optimize the program. The APP performance testing tool came into being based on the automation requirements of this process.
The Android operating system itself provides some official performance testing tools, including: HierarchyViewer for detecting layout complexity, power statistics in developer mode, DDMS-MAT for detecting memory leaks, etc.
However, these tools often need to be deployed and used separately, they are not well integrated, and the time and cost of installation, learning and use are relatively high.
There are also many automated tools for APP performance testing on the market. The mainstream third-party tools include: LeakCanary, Emmagee, iTest, GT, and the recently hyped PerfDog. These tools are compared in four aspects:
1. Test scope
Except for LeakCanary, which is only for memory leaks, all other tools can support the monitoring of most indicators in APP performance testing.
2. Ease of use
iTest/Emmagee/PerfDog is obviously better. These three tools can perform performance tests on APP without integrating SDK.
3. Platform Compatibility
GT and PerfDog support both Android and IOS systems, but PerfDog is better because ROOT and jailbreak are not required for testing on real devices.
4. Manufacturer influence
Both GT and PerfDog are from Tencent. GT is developed by the MIG special test team, and PerfDog is developed by Tencent's game department. We know that the requirements for game performance indicators are more demanding and strict than general application testing requirements.
So below we focus on the key features and usage of PerfDog.
2. Know PerfDog
Mobile performance testing has always been a headache for many performance testers: many tools are used for each test, ROOT/jailbreak is difficult, the accuracy of tools is doubtful, and multiple retests are required...etc. Based on this, Tencent developed a mobile performance testing tool, which was officially released on November 1, 2019, and is open for download - PerfDog, also known as performance dog.
PerfDog is a mobile full-platform iOS/Android performance testing tool platform, which can quickly locate and analyze performance problems, and improve the performance and quality of APP applications and games.
Eight features of PerfDog:
-
Full mobile platform - support Android/iOS mobile platforms, such as mobile phones, iPads, Watches, Android emulators, etc. Provide PerDog software Windows&Mac version.
-
No ROOT/Jailbreak required - No ROOT required for Android devices, no jailbreak required for iOS devices. Efficiently solve Android&iOS performance test analysis problems.
-
Data accuracy - the tool itself has an impact on the performance of the test equipment CPU < 1%, and has no impact on the frame rate. The accuracy and authenticity of performance data are guaranteed.
-
Data Integrity —Screenshot, FPS, Jank, FTime, CPU, GPU, Memery, Battery, NetWork, CTemp and other performance parameters can be provided, which can be easily obtained.
-
Extensive application —supports all APP applications, games, small programs, small games, H5, Web, etc.
-
Ease of use of tools - no installation required, plug and play, reducing cumbersome testing obstacles.
-
Cloud Kanban - Cloud-based Kanban that can be stored, visualized, edited, managed and downloaded anytime, anywhere Collect all test data using any version of PerfDog.
-
Teamwork - You can create tasks at any time, invite team members to join, support sharing and view team completion, and open an efficient collaboration mode.
3. Use of PerfDog
Step 1: Download and install
Enter the official website to download, register and log in.
Run the client after downloading, and start device connection after logging in, which is used for tools to obtain device information.
1.Android client
First set the phone to allow USB debugging (or enter developer mode), and connect the device to the computer with a USB cable. At this point, perfdog.apk will be automatically installed on the phone to run in the background and display real-time data during the test on the screen.
After installation, the apk will request permission. If you want to display real-time performance data on the screen, you must open the floating window permission.
2. iOS client
The iOS client does not require much operation by the user, it is plug and play.
iOS can use both USB test mode and WIFI mode. However, the WiFi mode also needs to use USB to connect the device first. After the tool recognizes and successfully connects to the device, select the WIFI mode in the drop-down box.
Step Two: Test Application Performance
Open PerfDog on the PC side, and after the successful connection with the mobile phone via USB, PerfDog can immediately identify the mobile phone model and various system parameters. After selecting the software to be tested, the monitoring data can be seen immediately. PerfDog displays three indicators by default, FPS/CPU Usage/Memory Usage. The indicator changes are displayed in the form of a graph, and the CPU Usage also shows the comparison between the total and the current APP consumption, and the data is very comprehensive. As shown below:
1. Label settings
Click "label1" on the top, you can set the label tab for different operations or interfaces, and use the label button to label the performance data.
2. Add comments
Double-click anywhere in the data to create annotations. To delete, move the mouse to the top of the comment and click again to delete.
3. Performance indicators
The indicators are very comprehensive. If the values displayed on the tool are not what you need, you can click the "+" sign in the lower right corner. After clicking, the selection item of the data value will pop up. If you check it, the data collection and recording of this item will be performed.
4. Data recording and saving
You can record certain data at any time by clicking the Record button in the upper right corner, and save it locally or directly upload it to the cloud server of PerfDog.
5. Data viewing
After the recording is completed, the data is saved as an excel file, which can be opened and viewed locally, or logged in to the web to review the test data. PerfDog's cloud and local use the same set of accounts, and you can view cloud data after logging in:
Click the data you need to view to enter the data report details page, which is very detailed. Each indicator is not only data, but also graphically displayed:
6. Test tasks can also be created and tracked in the cloud
Four. Summary
During the whole test process, I can't feel the impact of the test tool on the mobile phone at all, and it is very smooth. The data given on the official website: The Android platform is more concerned about the impact of the tool itself on the performance of the mobile phone. In order to test the rigor and comparability of the test, we locked the lowest frequency on the Mi 5 mobile phone. The test results with screenshots are as follows:
PerfDog (performance dog) performance impact: CPU accounts for 6%-10% (CPU accounts for 1-3% at normal frequency).
Performance impact of other tools: CPU accounts for 27%-33%
If you are also doing APP performance testing, you might as well try PerfDog, it will definitely help.
Official website PerfDog instruction manual:
https://bbs.perfdog.qq.com/article-detail.html?id=5
Reference documents:
https://testerhome.com/topics/21507
Here, I recommend you a group (785128166) to receive video learning materials. If you don’t want to experience the feeling of not being able to find materials during self-study, no one answering questions, and giving up after persisting for a few days, join us!