Jmeter usage tutorial, analysis of all actual combat tutorials from installation to HTTP pressure measurement, do not learn to regret series

As a development engineer, when we receive requirements, we usually analyze the needs, determine ideas, code, self-test, and then let the testers test. In the self-test step, as a developer, most of the time, it is to test whether the business process is correct and whether there are logical errors. If not, it is almost over. Then when the testers go to test, many times the testers just repeat the developer's steps, and more just expand the test cases, especially some boundary use cases. However, our interface is fine at 10 requests per second, but it does not mean that it is fine at 1000 requests per second. When the interface is deployed online, when the concurrency increases, many problems that did not appear in the testing phase will appear online. May have appeared.

Some companies have relatively large teams and can hire good testers to do various tests, but for small teams, testers may not know the difference between 10 requests per second and 10,000 requests per second. For us, in the case of ensuring the correctness of the business logic, we also need to do stress testing.

This article introduces a powerful stress testing tool - JMeter.

JMeter is a top-level project under Apache, take a look at the official introduction to it:

The Apache JMeter™ application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.

It probably means that JMeter is an open source software written in pure java, which is used to test variable behavior and performance. It was originally designed to test web applications, but has since been extended to other tests.

There are many applications/services/protocols that JMeter can support testing: Web (HTTP, HTTPS ), SOAP / REST Webservices, FTP, Database via JDBC, LDAP, Message-oriented middleware (MOM) via JMS, Mail - SMTP(S), POP3(S) and IMAP(S), Native commands or shell scripts, TCP, Java Objects. Anyway, some common things on the server side can be tested.

This article introduces HTTP pressure testing.

1. Download

You can go to the official website to find the download link or use the following link to download:

http://mirror.bit.edu.cn/apache//jmeter/binaries/apache-jmeter-5.2.1.zip

Unzip directly after downloading, the directory structure is as follows:

bin: executable script file

docs: Documentation of the api provided by JMeter

extras: extra files

lib: the jar package that JMeter itself depends on

licenses: the license of the jar package that JMeter depends on

printable_docs: JMeter introduction and manual

JMeter not only provides a GUI interface for users to use, but also provides a command line for users to operate. We first use JMeter under windows.

2. Preparation

2.1. Operating environment

JMeter5 requires at least JDK8, if your machine has not installed JDK, please install it first

2.2. Create interface

Here we use springboot to quickly build an interface, the main code is as follows:

@GetMapping(value = "/test")
public String performanceTest(@RequestParam(value="name", defaultValue="") String name) {
    log.info("进入测试，参数name的值为:{}", name);
    if (StringUtils.isEmpty(name)) {
        return "name cannot be null";
    } else {
        return RandomUtil.generateRandomString(16);
    }
}

Among them, generateRandomString is a method to generate random characters of a specified length.

Simply specify the log file in application.properties:

logging.file=spring.log

Then package and deploy to linux, the startup command is as follows:

java -jar stress-testing-0.0.1-SNAPSHOT.jar

Then use the curl command to access the interfaces with parameters and without parameters respectively, and you can also access them in the browser

3. Use JMeter to test HTTP

3.1. Create a test plan

We double-click to open bin/jmeter.bat, and wait for the JMeter GUI program to start. After success, it is as follows. The default language for the first startup is English, you can switch to Simplified Chinese in Options->Choose Language->Chinese(Simplified) in the menu bar

Right-click TestPlan->Add->Thread (User)->Thread Group, select Thread Group after completion, right-click on this thread group, Add->Sampler->HTTP request; then select HTTP request, in this HTTP request Right-click on it, Add->Listener->View Result Tree; select the HTTP request again, right-click on this HTTP request, Add->Listener->Aggregate Report. After completion, as shown in the figure

Let's explain these concepts:

• TestPlan: Test plan, equivalent to a project, what needs to be tested, and how to test is defined in a test plan
• Thread Group: thread group, equivalent to the number of simulated requests. A thread is equivalent to a user request
• View the result tree: monitor the status of each request when sending the request
• Aggregate report: summarize the data of a test

3.2. Configuration parameters

• Test Plan

Select Test Plan on the left, the name on the right is the name of the test plan, and the comments are equivalent to the comments in the code, there is nothing to say. Below is a thread group that runs independently. Multiple thread groups can be created in a test plan (currently we only have one). For example, the concurrency of our different interfaces is different. At this time, multiple threads can be created as needed. group, tested separately. Well, let's not modify it by default.

• Thread Group

Select the Thread Group on the left, and the thread group also has a name and comments. For example, we can fill in some information such as the concurrency level, anyway, it is for people to see. The following thread attributes are the core configuration. As mentioned earlier, a thread is equivalent to a user request. For example, fill in 10 for the number of threads, 5 for the ramp-up time, and 1 for the number of cycles, which means that 10 requests are sent within 5 seconds and executed once.

• HTTP request

Select the HTTP request on the left, here we focus on the two properties of Web server and HTTP request. The protocol is the protocol of the request, the default is http, ip fills in the server address, you can also fill in the domain name, the port number is 8080; because the test interface just written only supports GET requests, the method selects GET, the path is the request path of url, and the GET request Parameters can be directly carried on the path, or written to the following parameters, and the request parameters can be added by clicking the "Add" button.

It is also worth noting that the nodes on the left can be repeated in many cases, but the scope is different. For example, the current viewing result tree and aggregated report are created under HTTP requests, then the viewing result tree and aggregated report listen to the HTTP result. There can be multiple HTTP requests under a thread group. For example, if we have an H5 page to test, opening this page may request multiple interfaces at the same time. In this case, multiple HTTP requests need to be created. When viewing the result tree and aggregated reports created under the Thread Group, it is to monitor the results of all HTTP requests under this thread group.

3.3. Test

Click the green triangle button on the interface toolbar to start pressure testing, and you can see that the spring.log file will continuously output information

After waiting for the pressure test to end, we can choose to view the result tree on the left to see the request status of this round of testing

Select one of the requests to see the detailed information of the request on the right, including time, request data length, request address, etc.

Select Aggregate Report to see the data report of this round of testing

To explain this report:

Sample: the number of requests, the calculation formula is the number of threads * the number of cycles, if the thread group configuration is checked forever, then it is the actual number of requests sent when you stop the test. Average: the average response time, in milliseconds
. For example, the average response time here is 38 milliseconds
Median: The median of the response time, in milliseconds.
90% percentile: 90% of the response time is less than this value, in milliseconds. Here, 90% of the response times are less than 22 milliseconds
95% percentile: the meaning is similar to 90%
99% percentile: the meaning is similar to 90%
Minimum value: the minimum response time of this round of testing, in milliseconds.
Maximum value: The maximum response time of this round of testing, in milliseconds.
Abnormal %: The proportion of abnormal requests in this round of testing.
Throughput: It can be understood as QPS, which is the ability of the interface we tested to process requests. For example, here is an average of 2.2 requests per second
. Receive KB/Sec: the receiving rate of response data.
Send KB/Sec: the sending rate of request data
ends here. Do you think the interface we just wrote is perfect and the service is running? Have no flaws? Well, we select the Thread Group on the left side of the interface, then change the number of threads to 5000, and keep the others unchanged, simulating that 5000 users will visit our interface within 5 seconds. Then click the button of a gear and two brooms at the top of the interface to clear the result tree and aggregation report, and then click the green start button. The test results are as follows

You will find that when the concurrency increases, some requests will be abnormal. Then switch to the aggregation report, and you will find that the original average response time was only more than 30 milliseconds and directly increased to more than 5 seconds, and the abnormal rate also appeared. At this time, we need to do some tuning based on some error messages, which may be at the system level, or at the jvm level, or it may be a problem with the code itself. This is not what this article wants to talk about, so I won’t talk about it here.

Fourth, use variables in JMeter 

In the above example, when we fill in the HTTP configuration, the IP directly writes the IP address, but there is a problem. If our interface deploys nodes on different machines, we are testing the interfaces on different machines. You can’t change one after each test. If you want to test 30 interfaces on a service, wouldn’t it be a pain to change it? JMeter provides us with variables, which we can use in HTTP requests.

We right-click on Thread Group, Add -> Configuration Components -> User-Defined Variables, which can also be created under the test plan or HTTP request, so that the scope is the entire test plan or HTTP request. Then select the user-defined variable, and click the Add button on the right to add two variables: host and port:

The variables used in JMeter are referenced by ${}. For example, to reference the host variable, it is ${host}. Then we select the HTTP request and change the IP address and port to the form of the reference variable.

Then we click the start button again to test, and open the view result tree to see that the request is still sent to the address defined in the variable.

In this way, even if we have 30 HTTP requests, we can only modify the variable value in the user-defined variable.

In order to facilitate subsequent tests, we first change the number of threads to 500, then select Test Plan on the left, then click File -> Save Test Plan as, and then select the storage directory to get a file in jmx format, which will be used later document.

Five, JMeter command line use

When starting JMeter through bin/jmeter.bat, the following information will be output on the console:

From this prompt, we can know at least two pieces of information:

Do not use the GUI mode for testing, but use the CLI mode (actually using the command line)
to run the parameters can be changed Let me
talk about the second point first, because JMeter is written in pure Java and runs on the JVM, so its operation It will be controlled by JVM parameters. The default heap size is 1G (the initial value Xms and the maximum value Xmx are both 1G), and the maximum Metaspace is 256M (JDK8 has no concept of permanent generation, use Metaspace instead). There is a line (line 150) in bin/jmeter.bat that is used to set JVM parameters:

set HEAP=-Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m

For example, we can double the three parameters, and then start it again, and find that the JVM parameters have changed

But the official does not recommend that we directly modify the startup script, but suggests that we create a setenv.bat in the bin directory and set the running parameters in it (if you have configured the jvm parameters of tomcat, you will be familiar with this approach), We change the HEAP parameter of the jmeter.bat file to the original value, then create a setenv.bat file in the bin directory, and write the following content:

set HEAP=-Xms2g -Xmx2g -XX:MaxMetaspaceSize=512m

Restart JMeter and check its JVM parameters again as follows

The above is the operating parameter setting of JMeter. The purpose of saying so much is actually related to the first point. Our servers are basically Linux systems, and our servers should not be installed with a GUI operation interface, because this is not only useless for running services, but also consumes a lot of cpu and memory resources, and may also make the server useless. Stablize. This is why JMeter recommends that we do not use GUI to test. In this case, why does JMeter provide a GUI interface? Looking back at the startup console of JMeter, there is such a line of information:

jmeter -n -t [jmx file] -l [results file] -e -o [Path to web report folder]

Here we focus on a parameter that is -t, followed by the jmx file. This file is the jmx file saved above, which stores the configuration parameters such as thread group and HTTP request required for JMeter pressure testing. Open it and you can see that it is actually a file in xml format, which contains various parameters. If there is no GUI interface, it will be very troublesome for users to write such a configuration file by hand.

Having said so much, let's formally introduce the parameters of the jmeter command. The complete parameters of jmeter can be viewed through jmeter -?. Here are only a few commonly used parameters:

-n: Non-GUI mode, in fact, it is the command line mode

-t: followed by the test file (jmx file)

-l: followed by the log file, output the test process to the log file

-e: generate a report after the test ends

-o: test report storage directory, must be an empty directory

Use the command line test below:

After completion, a test report in html format will be generated in the specified directory. There is an index.html among them. When we open it, we can see a very beautiful graphical report, which can be shown to your boss.

6. Distributed testing 

The above is the operation on a machine, so there will be such a problem: when you set the number of threads to 10, execute it on a 4-core cpu machine, this machine simulates 10 threads of course No problem, but if you increase the number of threads to 10,000, on the surface this machine is also trying to simulate 10,000 threads for you, but its core number is only 4. In fact, the operating system is constantly switching threads at the bottom. To simulate these 10,000 threads, it takes time to switch threads, and it takes time to send requests. In this way, in addition to making the cpu usage rate of the test machine soar to 100%, you can also get an inaccurate or even wrong data. Because your machine actually has no way to send these 10,000 requests in 5 seconds. At this time, we need to use multiple machines to send requests to the interface machine at the same time, which is distributed testing.

Let me talk about the general principle. We prepare multiple machines, one of which is used as the master machine, and the rest are used as slave machines. The master machine is used to send instructions, and the slave machine executes them. The schematic diagram is as follows:

The requirements for these machines are as follows:

Close the firewall or open the corresponding port

on the same subnet

JMeter can access the interface of the test

The version of JMeter is consistent, and the version of JDK is also consistent, otherwise errors may occur

SSL must be set for RMI or turned off

If the above conditions are met, first execute the jmeter-server in the bin directory of each slave, then develop the JMeter bin/jmeter.properties file on the master machine, find the line remote_hosts=127.0.0.1, and change it to the slave intranet address , the addresses are separated by English commas, then open JMeter on the master machine, and test it like a single machine.

I don't have so many machines, so I won't demonstrate them here.

Finally, I would like to thank everyone who has read my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, you can take it away if you need it:

Software testing interview applet

The software test question bank maxed out by millions of people! ! ! Who is who knows! ! ! The most comprehensive quiz mini program on the whole network, you can use your mobile phone to do the quizzes, on the subway or on the bus, roll it up!

The following interview question sections are covered:

1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux

6. web, app, interface automation, 7. performance testing, 8. programming basics, 9. hr interview questions, 10. open test questions, 11. security testing, 12. computer basics

These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey. I hope it can help you too!