How SpringBoot uses JMH for performance testing

How SpringBoot uses JMH for performance testing

In SpringBoot applications, we can use JMH (Java Microbenchmark Harness) for performance testing. JMH is a tool dedicated to microbenchmarking, which allows us to test the performance of our code under different conditions. Using JMH allows us to test the performance of the code more accurately, thereby optimizing the code and improving the performance of the system.

In this article, we will introduce how to use JMH for performance testing and write a sample test case to test the performance of the code.

What is JMH

JMH is a tool dedicated to microbenchmarking, which allows us to test the performance of our code under different conditions. JMH was developed by the OpenJDK team and is one of the most commonly used performance testing tools in the Java language.

JMH can measure code execution time, method throughput, memory allocation, lock consumption and other indicators. JMH allows us to test the performance of the code more accurately, thereby optimizing the code and improving the performance of the system.

JMH provides a variety of test modes, including benchmark test mode, iterative test mode, black box test mode, etc. JMH also provides a variety of testing options, including the number of testing threads, testing times, testing time, etc.

Performance testing with JMH

In SpringBoot, we can use JMH for performance testing. JMH can be set up with the following steps:

1. import dependencies

Add the following dependencies to the pom.xml file:

<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>1.33</version>
    <scope>test</scope>
</dependency>

<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>1.33</version>
    <scope>test</scope>
</dependency>

2. write test class

Create a test class and add @Stateand @Benchmarkannotations. @StateAnnotations are used to specify the state of a test, and @Benchmarkannotations are used to specify a test method.

In the test class, we need to define the test state to be used in the test. We can @Statedefine test state using annotations.

@State(Scope.Thread)
public class MyBenchmark {
    
    
    private List<String> list;

    @Setup
    public void setup() {
    
    
        list = new ArrayList<>();
        for (int i = 0; i < 10000; i++) {
    
    
            list.add(UUID.randomUUID().toString());
        }
    }

    @TearDown
    public void tearDown() {
    
    
        list = null;
    }

    @Benchmark
    public void testSort() {
    
    
        Collections.sort(list);
    }
}

In the above example, we @State(Scope.Thread)defined a test state using annotations. We @Setupinitialize a list of 10,000 random strings in the method, and then @TearDownclear the list in the method. In @Benchmarkthe method, we use Collections.sort()the method to sort the list.

3. run test

In the test method, we use JMH for performance testing. We can run a performance test with the following code:

public class MyBenchmarkTest {
    
    
    @Test
    public void testMyBenchmark() throws Exception {
    
    
        Options options = new OptionsBuilder()
            .include(MyBenchmark.class.getSimpleName())
            .forks(1)
            .threads(1)
            .warmupIterations(5)
            .measurementIterations(5)
            .mode(Mode.AverageTime)
            .build();

        new Runner(options).run();
    }
}

In the above example, we use OptionsBuilder()the constructor to create the test option and use Runnerthe object to run the performance test. We use include()the method to specify the class to be tested, forks()the method to specify the number of test processes, the threads()method to specify the number of test threads, the warmupIterations()method to specify the number of warm-ups, the measurementIterations()method to specify the number of tests, and mode()the method to specify the test mode.

In the example, we use Mode.AverageTimethe pattern for testing. This mode runs the test multiple times and then calculates the average of the test results. Mode.SampleTimeThe pattern randomly selects some test cases to test and then calculates the sample mean of the test results. Mode.ThroughputMode tests the throughput of the code, which is the number of operations performed per unit of time.

4. view test results

After running the tests, we can view the test results. JMH will output a variety of test results, including test time, throughput, memory allocation and other indicators.

In our example, the test results output by JMH look like this:

Benchmark             Mode  Cnt   Score   Error  Units
MyBenchmark.testSort  avgt    5  4.517 ± 0.329  us/op

In the output result, Benchmarkthe column shows the name of the test method, Modethe column shows the test mode, Cntthe column shows the number of tests, Scorethe column shows the average value of the test result, Errorthe column shows the error range of the test result, and Unitsthe column shows the unit of the test result .

In our example, the name of the test method is MyBenchmark.testSort, the test mode is avgt, the number of tests is 5 times, the average value of the test results is 4.517 microseconds, the error range is 0.329 microseconds, and the unit is microseconds per operation.

Example test case

Below is an example test case to test the performance of a simple string concatenation method.

@State(Scope.Thread)
public class StringConcatBenchmark {
    
    
    private String str1;
    private String str2;

    @Setup
    public void setup() {
    
    
        str1 = "Hello";
        str2 = "World";
    }

    @TearDown
    public void tearDown() {
    
    
        str1 = null;
        str2 = null;
    }

    @Benchmark
    public String testStringConcat() {
    
    
        return str1 + " " + str2;
    }
}

In the above example, we @State(Scope.Thread)defined a test state using annotations. @SetupInitialize the two strings in the method, and then empty the strings in the @TearDownmethod. In @Benchmarkthe method, we use +the operator to concatenate two strings.

We can run a performance test with the following code:

public class StringConcatBenchmarkTest {
    
    
    @Test
    public void testStringConcatBenchmark() throws Exception {
    
    
        Options options = new OptionsBuilder()
            .include(StringConcatBenchmark.class.getSimpleName())
            .forks(1)
            .threads(1)
            .warmupIterations(5)
            .measurementIterations(5)
            .mode(Mode.AverageTime)
            .build();

        new Runner(options).run();
    }
}

After running the test, the test results output by JMH are as follows:

Benchmark                     Mode  Cnt   Score   Error  Units
StringConcatBenchmark.testStringConcat  avgt    5  3.729 ± 0.142  us/op

In the output results, we can see that the name of the test method is StringConcatBenchmark.testStringConcat, the test mode is avgt, the number of tests is 5 times, the average value of the test results is 3.729 microseconds, the error range is 0.142 microseconds, and the unit is microseconds per operation.

in conclusion

Using JMH for performance testing allows us to test the performance of the code more accurately, thereby optimizing the code and improving the performance of the system. In SpringBoot applications, we can use JMH for performance testing. When performing performance testing, we need to import JMH dependencies, write test classes, and use JMH for testing. After the test is completed, we can view the test results and optimize the code based on the test results.