Preface
Two performance indicators
business indicators;
technical indicators;
Usually we define the demand indicators of performance scenarios from two levels. They have a mapping relationship. Technical indicators cannot be separated from business indicators.
1. Concurrency
Narrow sense:
refers to performing the same operation (such as flash sale) at the same point in time.
Broad sense:
at the same point in time, any request initiated to the server (may be a different request)
will be received by the server within this point in time. to the request (regardless of whether it is the same request)
Scene analogy:
On the highway, how many cars pass through the same level at the same time, but they are not necessarily the same brand of cars.
Number of concurrent users (key points)
At the same time point, the number of users making requests, one user can make multiple requests;
the scenarios are not necessarily the same;
it is related to the CPU and response time;
Relationship with concurrency
Assuming there are 10 users and each user initiates 2 requests at the same time, then the number of concurrent requests received by the server is 20.
Performance test scenario 1:
Users with different identities visit different pages or initiate different requests (generalized concurrency);
observe CPU usage and response time;
Performance test scenario 2
: All users send the same request at the same time point (concurrency in a narrow sense);
observe CPU usage and response time;
The number of system users is
the cumulative number of registered users in the system, which may not necessarily be online;
you can also be offline after registration;
because user information is stored in the database, and the database data is stored in the disk, the number of system users is related to the disk space;
Performance test scenario
: Write a script to add many pieces of user information to the database;
Purpose: Test the system capacity to facilitate understanding of the system's maximum capacity;
In actual projects, when the system capacity is close to the maximum capacity, the system needs to expand its capacity (add disk space) , otherwise it will explode;
Number of online users.
Online users may initiate requests normally, or they may just hang up and do nothing, and may not necessarily do one thing at the same time.
Online users may be tourists (unregistered users) or system users (registered users). users);
number of online users ≠ number of concurrent users;
related to memory
Performance test scenario:
Use Jmeter to allow different users to continuously come online without going offline or making other requests to see the memory usage;
actual scenario: 12306 In the past, many users were online, and the response time would be very long;
Number of threads
In jmeter, the number of threads is equivalent to the number of concurrent users [related to CPU and response time]
2. Affairs
The process in which the client sends a request to the server and the server responds;
functions such as login, registration, and order placement all belong to one transaction;
one transaction may initiate multiple requests;
In jmeter related
jmerter, by default an interface request is a transaction;
but it also supports the integration of multiple interfaces into one transaction;
Note:
If a business or transaction has multiple interfaces, then the sum of the performance index values of multiple single interfaces ≠ the performance index value of the business or transaction
3. Response time
For performance testing, response time
is the time from initiating a request to receiving a request response;
it includes: Request Time and Response Time;
it is equivalent to: initiating request network transmission time + server processing time + database system processing time + return response network transmission time;
The response time perceived by the user includes
user client rendering time (extra this);
request/response data network transmission time;
application server processing time;
database system processing time;
Key points
: When doing performance testing, we must reduce the network transmission time as much as possible, so that the final RT will be infinitely close to the server processing time, so we must improve the network environment.
Transaction request response time
The time it takes to complete a single transaction, which may include multiple requests
If a user says that the application is slow, how should we analyze it? (For reference only)
Is it slow for a single user? Or is it slow for multiple users? Only our own application is slow? Or are all apps this slow?
If there is a network problem, which provider does the bandwidth use? Are all different operators stuck? Or only the merchant card where the user is located?
…wait wait wait
What is a reasonable response time?
The standards are: 2, 5,
8 2 seconds: Very good
5 seconds: Acceptable
8 seconds: Unacceptable
4. TPS (the most important indicator)
The number of transactions the server processes per second, the most important indicator of the server's processing capabilities.
Know how T is defined.
In different industries and businesses, the granularity of TPS definition may be different;
so in any case, the relevant parties of the business that need to do performance testing must know how your T is defined;
Defining the granularity of TPS The
granularity of TPS is generally defined according to the purpose of the scenario;
interface layer performance testing: T can be defined as the interface level;
business-level performance testing: T can be defined as each business step and complete business flow;
If you want to test interfaces 1, 2, and 3 individually, then T is the interface level
If you place an order from the user's perspective, then 1, 2, and 3 are all in one T, which is the business level.
Combined with the actual business design, the inventory service must be synchronous, while the points service can be asynchronous, so this order business can only be regarded as consisting of two interfaces 1 and 2, but interface 3 still needs to be monitored and analyzed.
Therefore, the definition of T in TPS in performance depends on the goal of the scenario and the role of T
Interface level script
- transaction start (interface 1)
interface 1 script
- transaction end (interface 1)
- transaction start (interface 2)
interface 2 script
- transaction end (interface 2)
- transaction start (interface 3)
interface 3 Script
- transaction end (interface 3)
Business-level interface layer script (that is, using interfaces to splice a complete business flow)
- Transaction start (Business A)
Interface 1 Script - Interface 2 (synchronous call)
Interface 1 Script - Interface 3 (asynchronous call)
- Transaction end ( Business A)
User-level script
- Transaction start (Business A)
Click 0 - Interface 1 Script - Interface 2 (synchronous call)
Click 0 - Interface 1 Script - Interface 3 (Asynchronous call)
- Transaction end (Business A)
Under normal circumstances, we will test one by one from top to bottom, so that the path is clearly executed and it is easy to locate the problem.
5、QPS
Query rate per second, the number of SQLs executed per second in the database.
One request may execute multiple SQLs.
Some companies may use QPS instead of TPS.
It is also an indicator of the server's processing capabilities, but it is not recommended.
A simple understanding of RPS
: the number of requests per second, the number of requests initiated by users from the client
Digging deeper: Regarding the number of requests, it also depends on which level the request is.
If a user clicks once, 3 HTTP Requests are sent, the order service is called 2 times, the inventory service is called 2 times, and the points service is called 1 time.
Q: How to calculate the number of requests
? A: 3+2+2+1 = 8?
No, it should be 3, because 3 Requests were issued, and the calling service will have a separate description for performance statistics.
6、HPS
Click rate, the number of clicks per second,
can be directly understood as the number of user clicks on the interface
. Generally used to describe HTTP Request in performance testing, it represents the number of HTTP requests sent per second, which is exactly the same concept as RPS. The larger the
HPS. The greater the pressure on the Server
7、CPS/CPM
The number of calls per second/minute
is usually used to describe the number of times the Service layer is called by other services per unit time.
Summary of TPS, QPS, RPS, HPS, and CPS.
There are many dimensions to measure the performance capabilities of a system, but it may not be too confusing if all five indicators are used to describe the system performance capabilities at the same time.
This can be done
by using TPS to uniformly describe the performance capabilities of the system, and the rest are described by adding restrictions at various levels.
For example: interface call 1000 Calls/s.
The team must clearly define the usage scenarios and meaning of the term.
Throughput (Throughput)
The number of requests processed by the network per unit time (transactions/s)
When there are no bottlenecks in the network, throughput ≈ TPS
Throughput
rate is the average rate of data transmitted over the network per unit time (kB/s)
Resource utilization:
The degree of usage of server resources, such as the CPU utilization, memory utilization, disk utilization, and network bandwidth utilization of the server (application, server), which generally does not
exceed 80%.
Think Time Thinking time
From a business perspective,
it refers to the time interval between each request when the user performs an operation.
For example: after adding to the shopping cart, how long does it take before the user clicks to place an order? How long does it take for a product to be added to the shopping cart?
From the perspective of performance testing, in
order to simulate the time interval between two user operations, Think Time is used to more realistically simulate the user's real operation.
It is related to user behavior, so it is user behavior rather than the number of users that should be analyzed.
Thank you to everyone who reads my article carefully. There is always a courtesy. Although it is not a very valuable thing, if you can use it, you can take it directly:
These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey. I hope it can also help you! Anyone in need Partners can click on the small card below to receive it