Previously in " What the Indicators in Performance Test Tell Us " briefly introduced the meaning of some basic performance indicators, and made it clear that the goal of our performance test is to find out the success rate of the request and not exceed the target request time. The maximum concurrency of our system . In this article, we do some practice and do a performance test from the perspective of programmer Xiao Zhang.
Do a simple performance test
First of all, let's simplify the problem a bit. Suppose Xiao Zhang received a website development task from the business manager. This website only designed one webpage, and the content is 10 pieces of information that the company will update and release every day.
Well, because of our assumption, the task is very simple. Xiao Zhang completed the function implementation by dividing by two, deploying the program to a server, and then telling the manager that it was done. The manager said that this website is very important. It is a project that the chairman and her sister-in-law personally grasped, and it is the face of the company. The chairman has repeatedly stressed that it can't be hung up like the 12306 website. Can you handle this program? Xiao Zhang said, let me do a performance test.
Xiao Zhang first asked the business manager, what is our maximum response time Latency , the business manager was a little forced, and said what you said. Xiao Zhang clears his throat. It means that the website user types in the URL and waits for the webpage to open. How many seconds can he wait at most? The business manager said, you can decide, so Xiao Zhang decided for 3 seconds.
After setting the maximum response time of 3 seconds, Xiao Zhang found a test tool on the Internet (we will talk about how many test tools there are in the following text), and started the performance test:
first test
Start 100 threads, each thread accesses a page 100 times, and performs stress testing.
As a result, 10,000 requests were completed in 32 seconds without failure. The average duration of each request was 32 milliseconds. In this test, what is the QPS and how much is the concurrency?
QPS : 10000 requests/32 seconds = 312.5 requests/second
Concurrency : 312.5 32=100
second test
Start 1000 threads, each thread accesses a page 100 times, and performs stress testing.
As a result, 100,000 requests were completed in 313 seconds without failure. The average duration of each request was 3.1 seconds. In this test, what is the QPS and how much is the concurrency?
QPS : 100000 requests/313 seconds = 319 requests/second
Concurrency : 319*3.1=990
Because the average response time in the second test has exceeded 3 seconds, Xiao Zhang looked at the number 990 and felt that the current maximum concurrency of the system should be around 1000. So I ran to tell the business manager that the website performance is up to 1000 concurrent. The business manager scratched his head and said that there are 3,000 people in our company, so if everyone takes a look, it will be dead. Xiao Zhang said that you think the user refreshes once, and it will be refreshed for the second time. The business manager said that it should take 5 seconds to complete the page after reading the page. Xiao Zhang said that the thinking time is about 5 seconds. When the thinking time is 5 seconds, the concurrency of 1000 means that it supports 5000 users to refresh. The business manager nodded and left with satisfaction.
This is the simplest simulation of a performance testing process.
to consider more
The above example is just to briefly understand the general process of performance testing, ignoring a lot of things, but for performance testing in real work, we need to consider more:
1. The bottleneck of the system
An important purpose of performance testing is to find out the performance bottleneck of the system. The purpose of finding the bottleneck is actually to optimize or ensure the possibility of optimization. From the perspective of software development, we have expectations for system bottlenecks. We expect bottlenecks to appear at the hardware level, not at the software level. If you find that the server's CPU, memory and other hardware indicators have not changed during the performance test, it means that the bottleneck must be in the network bandwidth and software, then our work may be to adjust the DB connection pool, OS operation For indicators such as the number of handles, our job is to adjust those that are well-adjusted, and finally adjust those that are not well-adjusted.
From the above two test results, although the average duration of each request in the second time exceeded 3.1 seconds, but in fact, compared with the first time, the QPS of the system did not increase, indicating that in the first test In fact, it has reached its bottleneck. The reality is that the above two results are actually the results of the author's local stress test on the Baidu website, and the bottleneck is actually the export bandwidth of the author's machine. Of course, this is not allowed in real work. The network elements are often ignored between the performance test client and the tested machine. The usual practice is to put them in a unified intranet environment.
Therefore, in the actual test process, we will not only care about the data of the test client like Xiao Zhang, but we must always monitor the performance indicators of the server and middleware to find out the system bottleneck and analyze and optimize to ensure that the bottleneck appears in us. Expect where he appears.
2. The granularity of pressure measurement
What Xiao Zhang is facing is actually the simplest kind of stress test, a single-machine and single-application test, that is, there is only one host and only one application, and the performance is very well estimated. However, the actual production environment is often a multi-host multi-application environment, and there are a large number of chain-like calling relationships. For example, the ordering service must be called first, and then the payment service must be called. In this case, it is necessary to start from a single machine and a single application, and gradually expand to the cluster test. The chain-type call relationship should also be changed from a single application test to a link test. For example, first test the ordering service and payment service separately. , after finding their maximum concurrency, build a scenario where both are tested, and test the maximum concurrency of the chain. Different software types often have different stress testing solutions, and each company is different, and there is no one-size-fits-all standard.
3. Pressure measurement data flow
When the granularity of the pressure measurement is determined and the actual pressure measurement is carried out, it is often necessary to simulate the real data traffic. For example, to really simulate user login (the pressure test environment often needs to change some codes for the pressure test), and simulate the user to place an order, this part is often the most workload part of the pressure test work, because it is necessary to prepare virtual user data, write For stress testing scripts, it is often necessary to modify the code of the target system for stress testing to ensure that the stress testing can be performed normally (for example, skip password login, disable IP verification, etc.). In the traditional software field, one or several test machines with good performance are often found, and the corresponding traffic can often be constructed by executing the test script. Even if there are requirements for user data, tens of thousands of user data simulations are basically sufficient. However, for large websites in the Internet field, the acquisition of pressure measurement traffic data is often a big problem. What if there is not so much traffic data. There are two common approaches:
- Online traffic playback: save the real user data through tcpcopy or tcpdump, and then playback it to the stress test environment to obtain stress
- Self-developed pressure test platform: develop a traffic platform by yourself, support the construction of traffic models, create virtual traffic, write pressure test scripts, and execute the pressure test scripts on hundreds of pressure machines to put pressure on the pressure test environment. In fact, it is the same as traditional software looking for machine simulation, but the scale is much larger, so it requires a high degree of automation. At present, the Alibaba PTS platform actually exports the internal pressure measurement capabilities of Alibaba to Alibaba Cloud for small and medium-sized enterprises to use.
4. Front-end performance
If you only test for WEB pages, Xiao Zhang's test actually ignores one element, the front end. The access to a web page is often not a simple request, but a group of requests. In addition to the html request, there are also interface requests, static resource requests, and so on. If you want to consider the performance loss, Zhang should take all requests into account and test the server. In real life, the author has seen the situation that the entire website cannot be accessed because the front-end request pictures too much and consumes a lot of bandwidth. Of course, if the dynamic and static separation has been done in the actual project, the service stress test can be performed separately for front-end resource acquisition. However, the front-end performance test also needs to consider the aspects of user experience, such as the loading time of the first screen, which belongs to the front-end field and has little to do with the server. The author will not go into details. You can click this link to learn more.
Summary of front-end performance optimization and testing tools