Why does .toString() seem to fix an OutOfMemoryError exception for StringBuilder?

JVon :

I am learning how to microbenchmark things with JMH. I started with something seemingly simple: string concatenation for StringBuilder vs String +=.

From my understanding, I should make a State object that contains an instance of StringBuilder because I don't want to benchmark its constructor (nor do I want to an empty one every iteration anyway). Same goes for the String += test - I want a String object in my State to be concatenated with new strings.

This is my code:

@State(Scope.Thread)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
public class Test {

    @State(Scope.Thread)
    public static class BenchmarkState {

        public StringBuilder    builder;
        public String           regularString;

        @Setup(Level.Iteration)
        public void setup() {
            builder         = new StringBuilder();
            regularString   = "";
        }

    }

    @Benchmark
    public String stringTest(BenchmarkState state) {
        state.regularString += "hello";
        return state.regularString;
    }

    @Benchmark
    public String stringBuilderTest(BenchmarkState state) {
        state.builder.append("hello");
        return state.builder.toString();
    }

    public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder()
                .include(Test.class.getSimpleName())
                .forks(1)
                .timeUnit(TimeUnit.MILLISECONDS)
                .mode(Mode.Throughput)
                .measurementTime(TimeValue.seconds(10))
                .build();

        new Runner(opt).run();
    }

}

It works, but I was thinking - I don't want to call .toString() at the end of every iteration. I am testing concatenation only. So I decided to remove it by just returning null instead.

But then, this happens during the first warmup iteration:

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3332)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
    at java.lang.StringBuilder.append(StringBuilder.java:136)

I understand that I would run out of memory pretty quickly if JMH is appending to StringBuilder as fast as it can, so I'm not surprised by the OutOfMemoryError issue. But I don't understand why does builder.toString() fix it.

So my questions are:

• Why does builder.toString() avoid an OutOfMemoryError issue? Doesn't StringBuilder still keep all the characters in memory regardless?

• Assuming that I do NOT want neither StringBuilder's constructor nor its .toString() method to be part of the benchmark, how do I properly write this test?

Andreas :

Calling toString() takes time, and generates garbage, requiring GC runs, further slowing down the code.

Since testing has a time limit, those slowdowns likely cause test to stop before it consumes all memory. If you increase the time limit, the code will likely fail with OOM even with the toString, it will just take a LOT longer.