Spring integration test consumes a lot of memory, uses a high number of duplicate threads in GradleWorkerMain

Erik Zivkovic :

I have a somewhat complicated Spring Boot app, with a large number of tests.

When running the tests, it seems to be accumulating a lot of threads, one of which there is multiple instances of and is called SimplePauseDetectorThread_0, which I traced down to this dependency

|    |    |    \--- io.micrometer:micrometer-core:1.1.1
|    |    |         +--- org.latencyutils:LatencyUtils:2.0.3

This seems to happen on Spring Boot 2.0.6 as well as 2.1.1.

A typical test may look like this:

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.MOCK)
@ActiveProfiles(profiles = {"test"})
public class MyTest {
[...]

My actuator config looks like this:

management.endpoints.enabled-by-default=false
management.endpoint.prometheus.enabled=true
management.endpoints.web.base-path=/
management.endpoints.web.exposure.include=prometheus
management.endpoints.web.path-mapping.prometheus=prometheus
spring.metrics.prometheus.enabled=true

See attached screenshot

Erik Zivkovic :

Snicoll from Pivotal helped me on GitHub, by suggesting that it was probably connected to context caching in the spring boot test framework.

If you have a large number of tests that are using the spring integration and a somewhat important number of context configurations (you've only shown one class), then one context per configuration will be created and I can see the number of threads increasing in that scenario.

He then pointed me to the relevant documentation, which states:

You can configure the maximum size from the command line or a build script by setting a JVM system property named spring.test.context.cache.maxSize. As an alternative, you can set the same property programmatically by using the SpringProperties API.

And org.springframework.core.SpringProperties states:

Reads a {@code spring.properties} file from the root of the Spring library classpath

Which leaves us with two ways of setting maxSize.

Option 1. Configure the gradle test task

Add a property to the gradle test task which will configure the GradleWorkerMain, in build.gradle:

test {
    jvmArgs "-Dspring.test.context.cache.maxSize=1"
}

If you have many subprojects you might want to use this option.

See Bonus below for a way to apply the setting to all your subprojects.

Option 2. Add a spring.properties to your test resources

You can write the setting in my-service/src/test/resources/spring.properties, like so:

spring.test.context.cache.maxSize=1

Conclusion

Now my tests are running nicely with less memory consumption, and fewer threads.

Bonus

This also resolves the issue with Gradle 5+ having workers that have 512MB max heap by default (instead of 25% of system RAM) - the subproject test suites no longer blow away all the available RAM which would cause the workers to OOM if I did not add a custom jvmargs with a larger heap in the test configuration of java projects. I can now run with "vanilla" heap size in the gradle worker.

If one does want to tweak the RAM available to Gradle tests, do something like this in the root build.gradle:

allprojects { project ->
    project.plugins.withId('java') {
        test {
            maxHeapSize = "1536M"
            // If you don't want to use spring.properties (or add other JVM args)
            jvmArgs "-Dspring.test.context.cache.maxSize=1"
        }
    }
    project.plugins.withId('java-library') {
        test {
            maxHeapSize = "1536M"
            // If you don't want to use spring.properties (or add other JVM args)
            jvmArgs "-Dspring.test.context.cache.maxSize=1"
        }
    }
}