Indicators in stress testing
1 Indicators in the stress test
1.1 TPS
TPS stands for Transactions Per Second, the number of transactions processed per second.
A transaction refers to the process in which a client sends a request to the server and the server responds (complete processing, that is, the client initiates the request to get the response) . The client starts timing when sending a request, and ends when receiving a response from the server, so as to calculate the time used and the number of completed transactions, and finally use this information to make an evaluation score. A transaction may correspond to multiple requests, you can refer to the transaction operation of the database .
1.2 QPS
QPS is the abbreviation of Queries Per Second, the number of queries that can be processed per second (complete processing, that is, the client initiates a request to get a response).
It is the number of queries that a server can respond to per second, and it is a measure of how much traffic a specific query server can handle within a specified time.
We can derive from its full English name that it means query. It turns out that on the Internet, the performance of a machine serving as a DNS server is often measured by the query rate per second. Corresponds to fetches/sec, which is the number of response requests per second.
Although it means query in name, in fact, it is now used to express the processing power of a single interface service in QPS (even if it is not a query operation).
1.3 Average processing time (RT)
RT: Response time, the average processing time required to process a request.
We generally also pay attention to the average processing time of 90% of the requests, because there may be extreme situations due to network conditions.
1.4 Number of concurrent users (concurrency)
The number of users requesting the interface to be tested per second.
1.5 Conversion relationship
QPS = 并发数/平均响应时间；
并发量 = QPS * 平均响应时间
For example, 3000 users (concurrent amount) access the interface to be tested at the same time. According to statistics on the user side, the average response time for 3000 users is 1188.538ms. So QPS=3000/1.188538s= 2524.11 q/s.
We can describe this test in this way, in the case of 3000 concurrency, the QPS is 2524.11, and the average response event is 1188.538ms
1.6 The difference between TPS and QPS
At the beginning of this question, I thought that the two should be the same thing, but when I saw their English names on
Zhihu , now I think: QPS can process the number of queries per second, but it is generally used for single service interfaces per second. The number of requests that can be processed.
The number of transactions processed by TPS per second. If the transaction is completed only for a single service interface, we can also consider it as QPS.
PS: There is also the concept of RPS request per second. The number of requests per second is similar to QPS and TPS under certain conditions.
2 Stress test method
We can use stress testing tools to simulate multiple users to stress test the system. With a certain total amount of requests, keep it unchanged, gradually increase the amount of concurrency, and observe the changes in QPS and the average response time.
For example, the total number of requests is 10,000, and then the QPS value under the condition of 100 concurrency is tested, and then 200, 300, 400, 500 and so on.
The throughput of a system is usually determined by two factors: TPS and the number of concurrency. These two values of each system have a relative limit value. Under the application scenario access pressure, as long as one item reaches the highest value of the system, the throughput of the system is If the pressure continues to increase, the throughput of the system will decrease instead. The reason is that the system is overloaded, and other consumption such as context switching, memory, etc. leads to a decrease in system performance. Here is a pressure measurement diagram using the ab tool.
It can be seen from the figure that the QPS has reached about 2500 when the concurrency is 2000, and the subsequent increase in the number of concurrency still remains at 2500, indicating that the QPS of the interface is 2500 under this configuration, that is, the capacity of the system can only be processed per second About 2500 requests, the subsequent increase in concurrency will only lead to an increase in the average response time. (PS: Because only 2500 requests can be processed per second, and there are 7000 concurrent requests at a time, it will naturally cause requests to accumulate, resulting in a longer average response time) We see that even after 14000, even QPS has begun to drop sharply, indicating that the system Overworked, resulting in a sharp decline in performance.
In general, we think that the average response time reaches a certain value, which is no longer acceptable.
3 Explanation of the name concept
1. QPS
Queries Per Second, the number of queries per second. The number of queries that can be responded to per second. QPS is a measure of how much traffic a specific query server handles within a specified period of time. On the Internet, the performance of a domain name system server machine is often measured by the query rate per second. The number of response requests per second is the maximum throughput capacity.
2. TPS
Abbreviation for Transactions Per Second, the number of transactions processed per second. A transaction refers to the process in which a client sends a request to the server and the server responds. The client starts timing when sending a request, and ends when receiving a response from the server, so as to calculate the time used and the number of completed transactions, and finally use this information to make an evaluation score.
The process of TPS includes: the client requests the server, the server internal processing, and the server returns to the client.
For example, accessing an Index page will request the server 3 times, including once html, once css, and once js, then visiting this page will generate a "T" and generate three "Qs".
3. RPS
RPS stands for Throughput Rate, which is the abbreviation of Requests Per Second. Throughput rate is a quantitative description of the concurrent processing capacity of a server, in reqs/s, which refers to the number of requests processed per unit time under a certain number of concurrent users. The maximum number of requests that can be processed per unit time under a certain number of concurrent users is called the maximum throughput rate.
Some people say that RPS is equivalent to QPS. In fact, it can be regarded as the same statistical method, but the name is different. RPS/QPS can be measured with the apache ab tool.