Summary of test veterans, detailed explanation of the three core indicators of performance testing, concurrency/TPS/response time...

Contents: Guide

foreword

Raise the question:
What the hell is concurrency?
How to judge that the system has reached a performance bottleneck?
Which ones should we pay attention to when monitoring so many indicators?

Let me give you a classic example: the story of the high-speed toll booth

Assuming that there is a high-speed toll booth now, there are 4 toll windows, and each window passes a car in an average of 5 seconds.
There are 2 ETC windows among them, and an average of 1 second passes through a car.
Now there are 100 vehicles arriving at the tollbooth at the same time, how long will it take to pass through completely? How long does it take on average for each vehicle to pass? How to improve passing efficiency and reduce passing time?

Taking the above example as an example, assuming it is a performance requirement, what knowledge points do you think of when you see it at first glance, and what factors should you consider?

If it were me, I would probably think of the following factors:
the toll booth is a server with a 4-core configuration, the official (or historical data shows) performance is the average response time = 5 seconds; 2
ETC windows, it may be the request to open The business (or technical) priority distinction is marked, and the specific scenario here should be considered;
100 vehicles arrive at the same time, which is the "concurrent number" simulated by the stress testing tool, but it is not actually a request processed by the server at the same time per unit time ;

The demand is asking about the total time-consuming and average response time of passing, and how to improve the passing efficiency and reduce the passing time. How to solve this problem?

What the hell is concurrency?

Strictly speaking, concurrency refers to how many requests the server receives per unit time, which is an important indicator to measure how much pressure the system is facing.

But the reality is that many students take common stress testing tools and use the number of concurrent threads/VU as the so-called concurrent number to understand.

Just like the high-speed toll station story above, you think the concurrency is 100 vehicles, but the real concurrency for the system is four toll windows.

Therefore, I suggest that students understand one thing when learning performance testing: the so-called simulation of concurrent numbers with tools is actually just to satisfy the action of putting the system under sufficient pressure.

How to understand TPS?

TPS, the official explanation is the number of transactions per second, which describes the process of the system completely processing an entire business request within a unit of time.

Taking the high-speed toll booth above as an example, there are 4 toll windows, and each window passes a car in an average of 5 seconds. There are 2 ETC windows among them, and an average of 1 second passes through a car.

What is the TPS of the entire toll station? Calculation method: (4 charging windows*1)/5 seconds=0.8.

Of course, business scenarios need to be distinguished here. If we talk about the ETC scenario alone, there are 2 charging windows, 1 per second, then TPS=2.

Do you understand?
In a real performance test, TPS needs to take into account specific business scenarios, and requires the request to be logically processed completely.

From a technical point of view, if it is a write request, you need to ensure that the database or cached data has been changed in line with business logic.

How to understand response time?

After talking about concurrency and TPS, let's talk about response time.
How to understand the response time? This should be understood from two dimensions.

client: The client initiates a request, transmits it to the server through the network, the server completes the logical processing, and returns the response to the client.
server: Start processing from receiving the request, to complete the logical processing and send it from the server, OK ends here, and a statistical interval is completed.

In the scene of changing to a high-speed toll station, when the car arrives at the toll window, I swipe the card and scan the QR code to pay, and then lift the lever to let the car go out until the next car comes in.

The time-consuming of this process is the so-called response time.

As for our common average response time and 99% response time, they are just statistical methods of different dimensions.

The relationship and influence among the three

How to improve the passing efficiency and reduce the passing time is the above problem.

Before solving this problem, we might as well substitute a real high-speed toll station. We imagine the following situations.
100 vehicles arrive at the same time, but there are only four toll windows, which need to be queued to enter in order, which is the queue mechanism; 100
vehicles arrive at the same time, but there are only four toll windows, in case the staff change shifts for lunch break, this is system maintenance;
100 The cars arrive at the same time, but there may be a traffic police road inspection. In fact, there are only 2 lanes to pass through the four toll windows. This is the current limit;
a car without ETC is going through the ETC lane. If it cannot pass, it needs to reverse and switch to other tolls. window, which is blocked or abnormal;

After reading the above situations, let's see how to understand the relationship and influence among the three.

Improving the passing efficiency can increase the charging window (server upgrade configuration, 4 cores to 8 cores);
improving the passing efficiency can allow more car owners to install ETC (technical optimization, all use cache);
improving the passing efficiency can be ensured through a certain mechanism The lane does not go wrong (abnormal handling, fast failure or release);
improve the passing efficiency, and can ensure the normal service of the toll window through 7*24 hours shift (service availability);

Is there any other way to improve pass efficiency and reduce pass time consuming?

Method:
Change 3 lanes of high-speed to 6 lanes (extend network bandwidth);
open several different toll stations (service expansion); distinguish
between the speed of large trucks and small passenger cars (splitting mechanism);
special toll stations for large trucks and small passenger cars (service group);

The real concurrency is the number of requests received and processed by the server per unit time, not the number of concurrent threads simulated by the tool.
When other conditions remain unchanged, the easiest way to increase TPS and reduce response time is to upgrade + expand + cache.
High concurrency/high performance/high availability actually means that more requests are processed in less time, and the service provides normal service for a long time.

The following is the most complete software test engineer learning knowledge architecture system diagram in 2023 that I compiled