7. Requested array size exceeds VM limit

7.1 Requested array size exceeds VM limit 概述

Java has a limit on the maximum array size your application can allocate. The exact limit is platform-dependent, but is usually somewhere between 1.0 and 2.1 billion elements.

When you face it java.lang.OutOfMemoryError: Requested array size exceeds VM limit, it means that the application crashed because it tried to allocate an array larger than the JVM can support.

7.2 Reasons

This error is thrown by the native code of the JVM. It occurs before allocating memory for an array, and the JVM performs a platform-dependent check: whether the data structure to be allocated is addressable on this platform. The Mistakes are not as common as you might first think.

The reason you rarely face this error is that Java arrays are indexed by type int. The largest positive integer in Java is: 2^31 -1 = 2,147,483,647. Platform dependent limits do come quite close to this number – for example on a MacBook Pro , Java 1.7, can initialize arrays to 2,147,483,645 or Integer.MAX_VALUE-2elements .

Adding 1 to the array length Integer.MAX_VALUE-1will cause this OutOfMemoryError:

Exception in thread “main” java.lang.OutOfMemoryError:
Requested array size exceeds VM limit

But sometimes this limit is not so high - on 32-bit Linux, OpenJDK 6, you will get an error when allocating an array of about 1.1 billion elements. To know the limit size of java.lang.OutOfMemoryError: Requested array size exceeds VM limityour specific environment, you can run the following small the test program.

7.3 Examples

To try to reproduce java.lang.OutOfMemoryError: Requested array size exceeds VM limitthe error, let's take a look at the following code:

for (int i=3; i>=0; i--) {
    try {
        int[] arr = new int[Integer.MAX_VALUE-i];
        System.out.format("Successfully initialized an array with %,d elements .\n", Integer.MAX_VALUE-i);
    } catch (Throwable t) {
        t.printStackTrace();
    }
}

The example loops four times, initializing a long array of primitives in each loop. The size of the array that the program is trying to initialize grows by one each iteration, eventually reaching an integer MAX_VALUE. You should now get output similar to the following when launching the snippet with Hotspot 7 on 64-bit Mac OS X:

java.lang.OutOfMemoryError: Java heap space
  at eu.plumbr.demo.ArraySize.main(ArraySize.java:8)
java.lang.OutOfMemoryError: Java heap space
  at eu.plumbr.demo.ArraySize.main(ArraySize.java:8)
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
  at eu.plumbr.demo.ArraySize.main(ArraySize.java:8)
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
  at eu.plumbr.demo.ArraySize.main(ArraySize.java:8)

Note that before the last 2 attempts are faced java.lang.OutOfMemoryError: Requested array size exceeds VM limit, the allocation failure reported the more common java.lang.OutOfMemoryError: Java heap spacemessage. This is because the 2^31-1 int you are trying to allocate requires 8G memory, which is much larger than the default memory of the JVM.

This example also illustrates how rare this error is - in order to see the VM reach the limit of the array size, you need to allocate a data that is just between the platform limit and Integer.MAX_INTthe 2 array lengths that meet the requirements: Integer.MAX_INT-1and Integer.MAX_INT.

7.4 Solutions

java.lang.OutOfMemoryError: Requested array size exceeds VM limitOccurs in the following 2 situations:

• Your array has grown too large, and the final size is between the platform limit Integer.MAX_INTand
• You are specifically trying to allocate an array larger than 2^31-1 elements to experiment with that limit.

In the first case, check your codebase to see if you really need such a large array. Maybe you can reduce the size of the array and be done with it. Or divide the array into smaller scatter points and load the required data into batches within the limits of your platform.

In the second case - remember that Java arrays are indexed by int. So you cannot have more than 2^31-1 arrays of elements when using standard data structures in the platform. In fact, in this case, the compiler has already stopped you when it announced "error: integer number too large" during compilation.

However, if you do work with really large datasets, you need to reconsider your options. You can load the data you need in smaller batches and still use standard Java tools, or you may be over Utilities. One way to achieve this is to look at sun.misc.Unsafethe class . This allows you to allocate memory directly as in C.