Three collection environments for metrics
In a specific performance test, metrics can be collected based on the following information:
❑Business environment (including business processes, customer and user behavior, stakeholder expectations, etc.);
❑Operating environment (including the technologies required for testing and how these technologies are used);
❑Technical environment
Note that tests in different fields have different requirements for metrics.
technical environment
1. The measurement indicators of technical environment performance testing vary according to different technical environments. Common technical environments include: browsers, mobile terminals, Internet of Things (IoT), desktop clients, servers, mainframes, databases, and networks. In addition to these contents, performance testing should also consider the characteristics of the environment in which the application software runs (such as embedded systems).
The corresponding metrics for the technical environment are as follows:
❑Response time, such as the response time of each transaction, the response time of each concurrent user, and the page loading time;
❑Resource utilization, such as CPU, memory, network bandwidth usage, network latency, available disk space, IO rate, idle and busy thread ratio;
❑Key transaction throughput rate, which uses a percentage to express the number of transactions that can be processed in a specific time period;
❑Batch processing time, such as waiting time, output time, database response time, completion time;
❑Number of errors affecting performance;
business environment
From a business or functional perspective, performance metrics can include the following categories:
❑Business processing efficiency, such as the execution speed of a complete business process, including normal, backup and abnormal use case processes or business scenarios;
❑The throughput of data, transactions and other work execution units, such as the order processing volume per hour and the increase of data rows per minute; ❑The compliance or violation rate of the service level agreement (SLA), such as the number of agreement violations per unit time;
❑Indicators related to scope of use