I. Introduction
Java JVM memory can generally be divided into three areas: heap (heap), stack (stack) and method area (method).
1.1 Heap area
1) All objects are stored, and each object contains information of a corresponding Class. The purpose of Class is to obtain operation instructions;
2) The JVM has only one heap area (heap) shared by all threads. The heap does not store basic types and object references, but only the objects themselves.
1.2 Stack area
1) Each thread contains a stack area, in which only objects of basic data types and references to custom objects (not objects) are stored, and objects are stored in the heap area;
2) The data (primitive types and object references) in each stack are private and cannot be accessed by other stacks;
3) The stack is divided into three parts: the basic type variable area, the execution environment context, and the operation instruction area (storing operation instructions).
1.3 Method area
1) Also called static area or permanent generation, like the heap, it is shared by all threads. The method area contains all Class and static variables;
2) The method area contains elements that are always unique in the whole program, such as Class and static variables;
3) The runtime constant pool is allocated in the method area of the Java virtual machine, but since the HotSpot virtual machine of JDK 1.7, the string constant pool originally placed in the permanent generation has been moved out.
Second, memory overflow analysis
The following code verification, it is recommended to use JDK 1.6, because since JDK 1.7, the HotSpot virtual machine has improved a lot, especially the method area (permanent generation).
2.1 Java heap overflow
The Java heap is used to store object instances. Therefore, as long as we continue to create objects and ensure that the objects are not cleared by the garbage collection mechanism, when the size of the objects in the heap exceeds the capacity limit of the maximum heap, heap memory overflow occurs.
The following code sets the size of the Java heap to 20MB and is not expandable (that is, set the minimum -Xms parameter and the maximum -Xmx parameter of the heap to be equal to 20MB to avoid automatic expansion of the heap); through the parameter -XX: +HeapDumpOnOutOfMemoryError allows the virtual machine to generate a snapshot of the current memory heap when a memory overflow exception occurs, for subsequent analysis; the function of -verbose:gc is to display information on the output device when memory reclamation occurs in the virtual machine.
package com.java.error; import java.util.ArrayList; import java.util.List; /** * VM Args:-verbose:gc -Xms20m -Xmx20m -XX:+HeapDumpOnOutOfMemoryError * Error:java.lang.OutOfMemoryError: Java heap space * @author moonxy * */ public class HeapOOM { public static void main(String[] args) { List<HeapOOM> list = new ArrayList<HeapOOM>(); while(true) { list.add(new HeapOOM()); } } }
After running, it displays as follows:
[GC (Allocation Failure) 5380K->3745K(19968K), 0.0042158 secs] [GC (Allocation Failure) 9287K->9710K(19968K), 0.0058399 secs] [Full GC (Ergonomics) 9710K->7589K(19968K), 0.1200134 secs] [Full GC (Ergonomics) 16387K->12869K(19968K), 0.1112792 secs] [Full GC (Ergonomics) 16428K->16382K(19968K), 0.1711686 secs] [Full GC (Allocation Failure) 16382K->16370K(19968K), 0.1371103 secs] java.lang.OutOfMemoryError: Java heap space Dumping heap to java_pid1204.hprof ... Heap dump file created [28024534 bytes in 0.077 secs] Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:3210) at java.util.Arrays.copyOf(Arrays.java:3181) at java.util.ArrayList.grow(ArrayList.java:261) at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235) at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227) at java.util.ArrayList.add(ArrayList.java:458) at com.java.error.OutOfMemoryJavaHeapSpaceTest.main(OutOfMemoryJavaHeapSpaceTest.java:19)
The console prints OutOfMemoryError, and the prompt is a memory overflow of the Java heap, and a dump file is generated. For Java heap overflow, a heap snapshot analysis is generally performed on the dump file through a memory image analysis tool (such as Eclipse Memory Analyzer) to confirm whether the objects in memory are necessary:
If the object is not necessary, it is a memory leak, and it is necessary to analyze why the object is not recycled;
If the object is really necessary to continue to exist, it is a memory overflow and needs to be avoided by increasing the size of the heap ( -Xms and -Xmx ).
Compare memory leaks and memory overflows here:
Memory leak memory leak: It means that after the program applies for memory, it cannot release the memory space that has been applied for. A memory leak does not seem to have a big impact, but the consequence of accumulation of memory leaks is memory overflow;
Out of memory: After the program applies for memory, there is not enough memory for the applicant to use or when the data of the long type is stored in the storage space of the int type, an OOM error occurs.
2.2 Virtual machine stack and native method stack overflow
Since the HotSpot virtual machine does not distinguish between the virtual machine stack and the local method stack, for HotSpot, the -Xoss parameter (set the local method stack size) is invalid, and the stack capacity is only set by -Xss .
In the Java Virtual Machine specification, there are two exceptions to this area:
If the total size of the stack frame when the thread is running exceeds the size of the virtual machine limit, a StackOverflowError will be thrown, which usually occurs when running recursively;
If the virtual machine stack is set to be dynamically expandable, and cannot apply for enough memory when expanding, an OutOfMemoryError will be thrown.
StackOverflowError
/** * VM Args：-Xss128k * Error:java.lang.StackOverflowError * @author moonxy */ public class JavaVMStackSOF { private int stackLength = 1; public void stackLeak() { stackLength++; stackLeak(); } public static void main(String[] args) throws Throwable { JavaVMStackSOF oom = new JavaVMStackSOF(); try { oom.stackLeak(); } catch (Throwable e) { System.out.println("stack length:" + oom.stackLength); throw e; } } }
After running, it displays as follows:
994 Exception in thread "main" java.lang.StackOverflowError at com.java.error.StackOverflowTest.StackOver(StackOverflowTest.java:15) at com.java.error.StackOverflowTest.StackOver(StackOverflowTest.java:16) at com.java.error.StackOverflowTest.StackOver(StackOverflowTest.java:16) at com.java.error.StackOverflowTest.StackOver(StackOverflowTest.java:16) at com.java.error.StackOverflowTest.StackOver(StackOverflowTest.java:16) ……
OutOfMemoryError
It is not recommended to perform the following verification, which is prone to system suspended animation, because in the virtual machine of the Windows platform, the Java thread is mapped to the kernel thread of the operating system.
/** * VM Args:-verbose:gc -Xss2m * Error:java.lang.OutOfMemoryError: unable to create native thread * @author moonxy * */ public class JavaVMStackOOM { private void dontStop() { while(true) { } } public void stackLeakByThread() { while(true) { Thread thread = new Thread(new Runnable(){ @Override public void run() { dontStop(); } }); thread.start(); } } public static void main(String[] args) { JavaVMStackOOM oom = new JavaVMStackOOM(); oom.stackLeakByThread(); } }
After running, it displays as follows:
java.lang.OutOfMemoryError: unable to create new native thread
Threads need to consume stack capacity, and there is a limit to the amount of memory the operating system can allocate to each process. Consider the following scenario: the total system memory is 6G, the JVM allocates 2G memory, of which the heap memory is allocated 1G, the method area (permanent generation) is allocated 512M, ignoring other smaller memory allocations, the rest is allocated to the virtual machine stack and the local method stack. The memory is only 512M, it is very likely that there is not enough free memory to create many threads. There may be java.lang.OutOfMemoryError: unable to create new native thread. If this is the case, consider reducing the heap size or appropriately reducing the stack allocation size per thread.
Threads consume memory, and if each thread consumes more memory, it will consume more memory overall. The default memory size per thread depends on the JVM implementation. You can use the -Xss parameter to limit the thread memory size and reduce overall memory consumption. For example, the JVM defaults to each thread occupying 1M memory, and the application has 500 threads, so it will consume 500M memory space. If in fact 256K of memory is enough for threads to run properly, configure -Xss256k, then 500 threads will only need to consume 125M of memory. (Note that if -Xss is set too low, a java.lang.StackOverflowError error will be generated)
2.3 Method area overflow
The method area stores data such as class information, constants, static variables, and code compiled by the JIT compiler loaded by the virtual machine. Before JDK1.7, HotSpot used the "permanent generation" to manage these data, that is, The data in the method area actually belongs to the heap memory and will occupy the heap memory.
Therefore: data size of method area < -XX:MaxPermSize < -Xmx
As long as we limit the size of the permanent generation (-XX:MaxPermSize), it is easy to overflow the method area. The method area size can be limited by setting -XX:PermSize and -XX:MaxPermSize.
/** * VM Args:-verbose:gc -XX:PermSize=10M -XX:MaxPermSize=10M * Error:java.lang.OutOfMemoryError: PermGen space * @author moonxy * */ public class OutOfMemoryPermGenSpaceTest { public static void main(String[] args) { List<String> oomPgsList = new ArrayList<String>(); int i = 0; while(true) { oomPgsList.add(String.valueOf(i++).intern()); } } }
The result after running is as follows:
Exception in thread "main" java.lang.OutOfMemoryError: PermGen space
If we execute the same code above in the JDK 1.7 environment, we will find that the while loop will continue to execute. The reason is that in JDK 1.7, HotSpot no longer puts constants in the permanent generation for management, but puts them in the Native Memory whose memory upper limit is the native memory. The advantage of this is to reduce the chance of memory overflow (no more -XX:MaxPermSize limit), and constants no longer occupy heap memory. This "de-permanent generation" approach has been in place since JDK 1.7. Finally, in JDK 1.8, it was completely implemented, the permanent generation was completely removed, and HotSpot added a new area called Mataspace, and provided -XX:MetaspaceSize and -XX:MaxMetaspaceSize parameters to set the running Java virtual machine The initial and maximum capacity of the Metaspace used.
2.4 Native direct memory overflow
Direct Memory is not part of the virtual machine's runtime data area, nor is it a memory area defined in the Java Virtual Machine Specification. But this part of the memory is also used frequently, and it may also cause OutOfMemoryError to appear.
The capacity of the native direct memory can be specified by -XX:MaxDirectMemorySize . If not specified, the default is the same as the maximum Java heap (specified by -Xmx).
An obvious feature of the direct memory overflow of this machine is that the dump file is very small, because the main objects are all in direct memory, and the exception information will not indicate in which area the memory overflow occurred, like this: java.lang.OutOfMemoryError
/** * * VM Args:-verbose:gc -XX:MaxDirectMemorySize * Error:java.lang.OutOfMemory * @author moonxy * */ public class DirectMemoryOOM { private static final int _1MB = 1024 * 1024; public static void main(String[] args) throws Exception { Field unsafeField = Unsafe.class.getDeclaredFields()[0]; unsafeField.setAccessible(true); Unsafe unsafe = (Unsafe) unsafeField.get(null); while (true) { unsafe.allocateMemory(_1MB); } } }
The results are as follows:
Exception in thread "main" java.lang.OutOfMemoryError
In summary, the following JVM parameters can be set:
-Xms256m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=512m
