This article is reproduced: https://blog.csdn.net/windforce828/article/details/104819055
What is OOM ?
OOM, the full name of "Out Of Memory", translated into Chinese means "out of memory", derived from java.lang.OutOfMemoryError. Take a look at the official description: Thrown when the Java Virtual Machine cannot allocate an object because it is out of memory, and no more memory could be made available by the garbage collector. It means that when the JVM does not have enough memory to When the object allocates space and the garbage collector has no space to reclaim, this error will be thrown (note: non-exception, because this problem is serious enough to be processed by the application).
Why is OOM?
Why is there no memory? There are two reasons:
1) Less allocated: For example, the memory available for the virtual machine itself (usually specified by the VM parameter at startup) is too small.
2) The application uses too much, and is not released when used up, which is wasted. This will cause memory leaks or memory overflows.
Memory leak : The memory used by the application is not released, causing the virtual machine to not use the memory again. At this time, this memory is leaked because the applicant does not use it and cannot be allocated by the virtual machine to others.
Memory overflow : The requested memory exceeds the memory size that the JVM can provide, and this is called overflow.
In the days when there was no automatic garbage collection, such as C language and C++ language, we must personally take charge of the memory application and release operations. If we apply for memory, we forget to release it after using it. For example, new in C++ but no delete , Then it may cause memory leaks. Occasional memory leaks may not cause problems, while a large number of memory leaks may cause memory overflow.
In the Java language, due to the existence of an automatic garbage collection mechanism, we generally do not need to actively release the memory occupied by unused objects, that is, theoretically, there is no "memory leak". However, if the coding is improper, for example, the reference of an object is placed in the global Map, although the method is over, the garbage collector will reclaim the memory according to the reference of the object, so that the object cannot be reclaimed in time. If this situation occurs too many times, it will cause memory overflow, such as the caching mechanism often used in the system. Memory leaks in Java, which are different from forgetting to delete in C++, are often caused by logical leaks.
When the JVM memory is severely insufficient, a java.lang.OutOfMemoryError will be thrown. This article summarizes common OOM causes and their solutions, as shown in the figure below.
1. Java heap space
When the heap memory (Heap Space) does not have enough space to store newly created objects, java.lang.OutOfMemoryError: Javaheap space error will be thrown (according to actual production experience, you can configure the key to OutOfMemoryError in the program log Word warning, once found, deal with it immediately).
Cause analysis
Common causes of Javaheap space errors can be divided into the following categories:
Request to create a very large object, usually a large array.
The amount of visits/data that exceed expectations is usually caused by a surge in upstream system request traffic, which is common in various promotional/seckill activities. You can check whether there is a sharp peak in combination with business traffic indicators.
Excessive use of the finalizer (Finalizer), the object is not immediately GC.
Memory leak (Memory Leak), a large number of object references are not released, JVM can not automatically reclaim them, common in the use of File and other resources are not reclaimed.
solution
In most cases, you usually only need to increase the JVM heap memory space with the -Xmx parameter. If still not resolved, you can refer to the following situations for further processing:
If it is a very large object, you can check its rationality, such as whether you have queried all the results of the database at once without restricting the number of results.
If it is the peak pressure of the business, consider adding machine resources or reducing the current limit.
If it is a memory leak, you need to find the held object and modify the code design, such as closing the unreleased connection.
2. GC overhead limit exceeded
When the Java process spends more than 98% of the time to perform GC, but only recovers less than 2% of the memory, and this action is repeated 5 times in a row, java.lang.OutOfMemoryError: GC overhead will be thrown limit exceeded error. Simply put, the application has basically exhausted all available memory, and the GC cannot reclaim it.
The cause and solution of this kind of problem are very similar to Javaheap space, you can refer to the above.
3. Permgen space
This error indicates that the Permanent Generation is full, usually because the number of loaded classes is too large or the size is too large.
Cause Analysis
Permanent generation storage objects mainly include the following categories:
The class definition loaded/cached in the memory, including the name, field, method, and bytecode of the class; the
constant pool;
the class associated with the object array/type array;
the class information optimized by the JIT compiler.
The usage of PermGen is positively related to the number/size of classes loaded into memory.
solution
Depending on the timing of Permgen space's error report, different solutions can be used, as shown below:
An error is reported when the program starts, modify the -XX:MaxPermSize startup parameter, and increase the permanent generation space.
An error is reported when the application is redeployed. It is likely that no application has not restarted, causing multiple copies of the class information to be loaded. Just restart the JVM to solve it.
An error is reported during runtime, and the application may dynamically create a large number of classes, and the life cycle of these classes is very short, but the JVM will not unload classes by default. You can set the two parameters -XX:+CMSClassUnloadingEnabled and -XX:+UseConcMarkSweepGC to allow the JVM to uninstall class.
If the above method cannot be solved, you can use the jmap command to dump the memory object jmap-dump:format=b,file=dump.hprof, and then use the Eclipse MAT https://www.eclipse.org/mat function to analyze the most expensive classloader and Repeat class.
4. Metaspace
JDK 1.8 uses Metaspace to replace Permanent Generation. This error means that Metaspace is full, usually because the number of loaded classes is too large or the size is too large.
The causes and solutions for such problems are very similar to Permgenspace, you can refer to the above. It is important to note that the startup parameter to adjust the Metaspace space size is -XX:MaxMetaspaceSize.
5. Unable to create new native thread
Each Java thread needs to occupy a certain amount of memory space. When the JVM requests the underlying operating system to create a new native thread, such an error will be reported if there is not enough resource allocation.
Cause Analysis
When JVM fails to request the OS to create a native thread, it will throw Unableto createnewnativethread. Common reasons include the following:
The number of threads exceeds the ulimit limit of the maximum number of threads of the operating system; the
number of threads exceeds kernel.pid_max (only restart); the
native memory is insufficient;
the common process of this problem mainly includes the following steps:
The application inside the JVM requests to create a new Java thread; the
JVM native method delegates the request and requests the operating system to create a native thread; the
operating system tries to create a new native thread and allocates memory for it;
if the operation The virtual memory of the system has been exhausted or is restricted by the address space of the 32-bit process, the operating system will reject this native memory allocation;
JVM will throw java.lang.OutOfMemoryError: Unableto createnewnativethread error.
solution
Upgrade the configuration to provide more memory for the machine;
reduce the size of the Java Heap Space;
fix the thread leakage problem of the application;
limit the thread pool size;
use the -Xss parameter to reduce the size of the thread stack;
increase the maximum number of threads at the OS level: execute ulimia-a View the maximum thread limit, and use ulimit-u xxx to adjust the maximum thread limit.
ulimit -a… Omit part of the content… max user processes (-u) 16384
6. Out of swap space?
This error indicates that all available virtual memory has been exhausted. Virtual Memory consists of two parts: Physical Memory and Swap Space. When the virtual memory requested by the runtime program overflows, it will report an Outof swap space? error.
Cause Analysis
Common reasons for this error include the following categories:
Insufficient address space;
physical memory has been exhausted;
native leak of the application, such as continuous application of local memory, but not release.
Execute the jmap-histo:live command to force Full GC to be executed; if the memory drops significantly after several executions, it is basically confirmed as a Direct ByteBuffer problem.
solution
The following solutions can be taken according to the cause of the error:
Upgrade the address space to 64 bit;
use Arthas to check whether it is an Inflater/Deflater decompression problem, and if so, call the end method explicitly.
For Direct ByteBuffer problems, the threshold can be lowered through the startup parameter -XX:MaxDirectMemorySize.
Upgrade server configuration/isolate deployment to avoid contention.
7. Kill process or sacrifice child
There is a Kernel Job called Out of Memory Killer, which "kills" certain processes when the available memory is extremely low. OOM Killer will score all processes, and then "kill" the processes with lower scores. For specific scoring rules, please refer to Surviving the Linux OOM Killer.
Unlike other OOM errors, the Killprocessorsacrifice child error is not triggered by the JVM level, but by the operating system level.
Cause Analysis
By default, the Linux kernel allows the total amount of memory requested by the process to be greater than the available memory of the system, and this "shift peak multiplexing" method can make more effective use of system resources.
However, this approach will inevitably bring certain "oversold" risks. For example, some processes continue to occupy system memory, and then other processes have no available memory. At this time, the system will automatically activate OOM Killer, look for processes with low scores, and "kill" them to release memory resources.
solution
Upgrade server configuration/isolate deployment to avoid contention.
OOM Killer tuning.
8. Requested array size exceeds VM limit
JVM limits the maximum length of the array. This error indicates that the array requested by the program exceeds the maximum length limit.
Before allocating memory for the array, the JVM checks whether the data structure to be allocated is addressable in the system, usually Integer.MAX_VALUE-2.
This type of problem is relatively rare, and it is usually necessary to check the code to confirm whether the business needs to create such a large array and whether it can be divided into multiple blocks and executed in batches.
9. Direct buffer memory
ava allows applications to directly access off-heap memory through Direct ByteBuffer. Many high-performance programs implement high-speed IO through Direct ByteBuffer combined with Memory Mapped File.
Cause Analysis
The default size of Direct ByteBuffer is 64 MB. Once the limit is exceeded, a Directbuffer memory error will be thrown.
solution
Java can only use Direct ByteBuffer through the ByteBuffer.allocateDirect method. Therefore, this method can be intercepted by online diagnostic tools such as Arthas for troubleshooting.
Check whether NIO is used directly or indirectly, such as netty, jetty, etc.
Adjust the upper limit of Direct ByteBuffer through the startup parameter -XX:MaxDirectMemorySize.
Check whether the JVM parameter has the -XX:+DisableExplicitGC option, if there is, remove it, because this parameter will invalidate System.gc().
Check the off-heap memory usage code to confirm whether there is a memory leak; or call the clean() method of sun.misc.Cleaner through reflection to actively release the memory space held by Direct ByteBuffer.
The memory capacity is indeed insufficient, upgrade the configuration.