In early November, Tencent official propaganda of a message, Tencent WeTest star -PerfDog tool for the global release. Officer Xuanjie Shao as follows: https://wetest.qq.com/lab/view/475.html . When I saw the news, feeling a kind of eye-opening, moving the end of the performance test it could be so simple. In her spare time today, came a wave of Exploration, simple experience a lot.
Performance data collection software
We at first to understand the tool by which performance data can be collected:
PerfDog support mobile platform for all applications (games, APP application, browser, applets, etc.) and the Android emulator, supports the use of desktop applications PerfDog run on Windows and Mac machines. To obtain performance parameters are as follows in iOS and Android platforms:
iOS platform (aligned in line with the official Apple Xcode tool parameters)
- Screenshot
- FPS (within 1 second of the game screen or application interface real average refreshes, commonly known as the frame rate / FPS)
1) Avg (FPS): average frame rate (over a period of time the average FPS)
2) Var (FPS): frame rate variance (paragraph time variance FPS)
. 3) the drop (FPS): the number of drop frames (FPS hourly average of two adjacent points is decreased more than 8 times) - Jank (within 1s .iOS9.1 following system Caton number is not supported FramePacing smoothness of the statistical principle similar to Android and iOS Jank Caton high FPS frame rate does not reflect smooth or not Caton such as:... FPS 50 frame, before rendering a 200ms, 800ms after rendering 49, although the frame rate of 50, but still feel very Caton, while the low frame rate FPS, does not mean Caton, such as non-uniform card suddenly FPS is 15. so, FPS average frame rate with no direct relationship Caton)
PerfDog calculation method: satisfy both conditions are considered a Caton Jank.
1, the current frame is time-consuming> first three frames took an average of two times.
2, the current frame Processed> Processed two film frames (1000ms / 24 * 2 = 84ms ).
Satisfy both conditions are considered to be a serious Caton BigJank.
1, the current frame is time-consuming> first three frames took an average of two times.
2, the current frame Processed> Processed three film frames (1000ms / 24 * 3 = 125ms ).
Computing ideas: consider visual inertia, assuming an average of three frames before the reference time-consuming, as vsync interval, twice vsync no new render screen refresh, is considered to be a potential Caton, that is to say the next frame time-consuming the first three frames greater than 2 times the average frame time-consuming, it is considered a potential Caton. Meet single frame while consuming more than twice the film frame took 1000ms / 24 * 2 (because the human eye can discern the picture below the 24 discontinuity), then that is a real Caton. If the single frame while consuming more than 3 times the film frame time-consuming, is considered to be a serious Caton.
Notes: Why is twice vsync? GPU typically 3-buffered buffer, occupied a current frame buffer, i.e., the remaining second buffer buffer, the human eye typically can tolerate two delay. Why are two film frame time-consuming? Less than 24 screen, the human eye can perceive discontinuity screen, movies are generally 24. That film frame took 1000ms / 24 = 41.67ms, two film frame is time-consuming 41.67ms * 2, three film frame is time-consuming 41.67ms * 3.
1) BigJank: the number of 1s in the Dayton serious card
2) Jank (/ 10min): an average of every 10 minutes Caton times.
3) BigJank (/ 10min): on average every 10 minutes, the number of serious Caton - FTIME (vertical frame picture display interval, i.e. that frame time consuming, iOS9.1 the system is not supported.)
. 1) Avg. (FTIME): Processed average frame
2) Delta (FTime): delta Processed (per time difference> h between two times of 100ms) - CPU Usage (Total Machine / App process, consistent statistical results together Xcode)
- Memory (statistical FootPrint, Note:. OOM and FootPrint related, regardless of the system model and only about RAM, 1G memory machine .FootPrint as more than 650MB, triggered OOM). By iOS platform restrictions, temporarily unable to obtain memory ios10 and following systems. Increase in subsequent versions. As do performance testing, recommended to upgrade the system version iOS
- Xcode Memory (XCode Debug Gauges statistical way that the XCode Memory). By iOS platform restrictions, temporarily unable to get Xcode Memory ios10 and following systems. Increase in subsequent versions. As do performance testing, recommended to upgrade the system version iOS
- Real Memory (Xcode Instrument statistical way that the Real Memory, the actual occupation of physical memory Note: Physical memory and system policies related to little attention meaning)
- Virtual Memory (virtual memory)
- Wakeups (thread wakes number). Note: more than 150 process will very likely be the system kill
- Cswitch (context switch test). Note: Single-core process will be more than 14,000 systems Kill
- The Utilization the GPU (the Render / Tilter / Device)
. 1) the Render: renderer utilization (pixel shading stage, if a high proportion, that is the bottleneck stage PS, Shader texture size or too complex, complicated sample)
2) Tilter: Tilter utilization (vertex shader processing stage, if the proportion is high, VS is the reason described bottlenecks stage, too many number of vertices, etc.)
. 3) device: utilization (GPU overall utilization) - Network (Recv / Send, a test target process flow, and Xcode results are consistent)
- Battery Power (real machine current Current, Voltage Voltage, Power Power) (Note: Xcode Instrument and the results are consistent)
- Log (system debug log information)
- Screenshot
- FPS (within 1 second of the game screen or application interface real average refreshes, commonly known as the frame rate / FPS)
1) Avg (FPS): average frame rate (over a period of time the average FPS)
2) Var (FPS): frame rate variance (paragraph time variance FPS)
. 3) the drop (FPS): the number of drop frames (FPS hourly average of two adjacent points is decreased more than 8 times) - Jank (. 1s the number Caton explanation as iOS platform described)
. 1) BigJank: within 1s serious Caton number
2) Jank (/ 10 min): the average every 10 minutes Caton number
3) BigJank (/ 10 min): on average every 10 minutes, the number of serious Caton - FTIME (vertical frame picture display interval, i.e. that frame time consuming)
. 1) Avg. (FTIME): Processed average frame
2) Delta (FTime): delta Processed (average difference between two hour time> 100ms number ) - CPU Usage (Total Machine / App target process, statistics and Android Studio Profiler consistent)
- CPU Clock (each CPU core frequency and utilization)
- Memory (PSS Memory, consistent statistical results and Android Java API standard results are also consistent with Meminfo Note: Part of Samsung machine system modifies the Meminfo underlying statistical methods, resulting in inconsistent Meminfo and Java AP statistics, the new Samsung machine has been repaired)
- Swap Memory (Swap Memory)
- Virtual Memory
- Memory Detail(NativePSS、GFX、GL、Unknown)
- GPU Usage (currently only supports Qualcomm chip phone)
- GPU Frequency (currently only supports Qualcomm chip phone)
- Network(Recv/Send)
- CTemp (CPU temperature)
- Battery Power (Current Current, Voltage Voltage, Power Power) (Note: the test instrument error <about 3%)
- Log (system debug log information)
From the official website using the above document. We understand the parameters, we actually operate it. For presentation tool, there on the web, I combined with their practical experience, for example.
The basic process used
Use in their own practice, the basic process is as follows:
1. Register an account (after only registered account to download the installation package)
2. Download and extract the installation package
3. Create a test project in the background perfdog
4. Open the executable file PerfDog.exe
5. Use a registered account login
6. Use usb phone and computer connections (not lock screen, turn on debug mode)
7. Select the connected mode (or USB WiFi)
8. Select the application list app
9. The configuration data to be monitored
10. The start recording data
11. The operation corresponds to app
12. The data recording is stopped (not less than 10S)
13. upload recorded data
14. enter perfdog backstage view performance data
Process Introduction
Not about the five steps before, we know everything. Let's start from the sixth step, I use the ios device.
Connected devices
iOS: plug and play, users do not need to do anything.
Android: There are two modes, the non-installation mode and installation mode.
-
. A non-mounting mode:
Phone plug and play, without any setup and installation, very simple to use, but there is no real-time performance data displayed on the phone screen.
-
. B installation mode:
PerfDog.apk need to automatically install on the phone, real-time performance data displayed on the phone screen. (Please open the Debug debugging mode that allows USB installation and PerfDog floating window management authority), start the PC version PerfDog.exe, PUSH is automatically installed on the phone PerfDog.apk specific installation similar to the various mobile phone manufacturers to install prompts to install third-party APP that is can. (Note: Since many cell phone installation requires account password, making it impossible to install automatically, if the automatic installation fails, the installation files PerfDog.apk release to the current folder, you can manually install PerfDog.apk).
Highlights here at the next Android platform and LMK Swap significance of these two parameters,:
LMK: OOM process memory size has nothing to do with the game on the Android platform, the system is primarily related to the remaining physical memory. The remaining physical memory is less than the system LMK, will cause OOM.
Swap: System process used zram / vnswap memory compression technology. Different phone systems enable Swap memeroy different sizes.
Test Mode
Connected to the computer via usb, the following interface, the test mode can be selected:
USB mode test:
USB cable, select the USB icon USB device mode test (test harness the power mode without any sense) in the list of devices.
WIFI mode test (test power):
After USB connection, select the icon WIFI WIFI device in the device list mode test. After detecting successful connection WIFI, unplug the USB cable. (Note: the need to test PC and mobile phone connected to the same WIFI, WIFI detection after a successful connection, disconnect the USB cable under test phone (power plug wire mode test without any meaning)).
In practice, USB and WiFi mode I have to use. After selecting the mode, the interface will show detailed information about the device, as follows:
Select Test Application
After selecting the mode, you can select the application you want to test the (all current app phone can be selected), the following page:
Select the corresponding application under test, corresponding to the operated App, the interface is shown below:
Precautions: Android platform, setup mode, the phone has the upper left corner of the screen to display real-time performance data (Android phone please open PerfDog floating window management authority, it would not show the performance parameters on the phone).
Open permissions Suspension
Android interface device performance parameters are shown below:
Features
1. Performance parameters
Performance parameters may be disposed in the interface, can click on the + interface, as follows:
① Click parameters corresponding entry, darken, chart data will be displayed in the interface
② check parameter corresponding entry indicates that the data need to be collected
2. Record Keeping
Click the Start button on the right side of the blue, it means that the data is recorded as follows:
Note that: the recording time can not be less than 10S. Less than 10S, it will prompt the following information:
After clicking the button, recording will stop recording and save the data as follows:
2.1 submit records to perfdog background
Can change the name, click on confirm, the data is uploaded to the background perfdog, as follows:
You can view the detailed performance data as follows:
Save records to a local 2.2
Check the Save button, the data is stored locally, as follows:
You can open the excel file to view the corresponding performance data:
3. Data Playback
Clicking on the folder perfdog interface button to select the corresponding local data, i.e., the recording can be played back, the operation is as follows:
The results can be viewed in playback interface, as follows:
4. annotation and Calibration
Double-click the left mouse button to increase the annotation, double-click again to cancel comment.
Click the left mouse button, increase the calibration, click again to re-calibration.
Comments and shown to increase the interface as calibration, annotations for the red, purple as calibration:
5. Parametric Studies
5.1 Statistics
Data can select a time period, statistics, as follows:
5.2 statistical analysis of the performance parameters set threshold
In perfdog Setting interface, can be configured as follows:
5.3 Saving marquee data
After a certain time period of the data marquee, fragments can be saved separately in the box selected range, to the right, as follows:
6. Label scene
Through the label tab button to play performance data, double click region may be modified corresponding to the color area of the tag name
7. Logging
In perfdog interface to view the corresponding log may be provided to view the log level, as follows:
WIFI mode when trying to find log button can not be checked.
8. Stop function
停止测试应用，不需要拔掉数据线，或者断开连接，在选择应用的界面中，选择NULL即可，如下：
9.截图录屏
连接安卓设备，并使用安装模式，可配置截屏参数，如下：
界面就会记录操作的过程，如下所示：
如此记录是不是很明了？但这种用法会影响性能参数，实际用途中不推荐。如果觉得新鲜，可以尝试使用即可。
PerfDog后台使用
1.邀请人员
可以邀请对应人员一起维护测试项目
2.数据共享
数据共享后，可以在任务数据中查看明细，可按android、ios区分，以及app包的版本，设备版本来查看。
使用注意点
1.设备连接
iOS： 若PerfDog检测不到连接手机或无法测试，请先安装确保最新iTunes是否能连上手机。
Android： 请开启手机Debug调试模式及允许USB安装。
2.截图记录影响性能
截屏记录影响性能(整体FPS影响<=1。小米5:CPU=1%左右。IPhone7P:CPU<2%)，若无需请不要开启截屏。
总结
使用PerfDog工具下来，整体有以下几点感受。
1.对性能指标的测试，更加便捷；
2.易操作
3.记录支持回放
4.数据便于管理与查看
PerfDog工具是款不错的性能测试工具，点赞一波。
最后，附上官方的操作手册：https://perfdog.qq.com/support