【JVM】Memory overflow

Reasons for memory overflow
    In the production environment, the tomcat memory setting is not good, and memory overflow is easy to occur. The reasons for the memory are different, and of course the processing methods are also different.

    Here is a summary based on the situation and relevant information that we usually encounter. There are three common cases:

    1.OutOfMemoryError: Java heap space

    2.OutOfMemoryError: PermGen space

    3.OutOfMemoryError: unable to create new native thread.

    Tomcat memory overflow solution

    For the first two cases, there is no memory in the application itself In the case of leakage, it can be solved by setting the tomcat jvm parameters. (-Xms -Xmx -XX:PermSize -XX:MaxPermSize)

    The last one may need to adjust the operating system and tomcat jvm parameters at the same time to achieve the purpose.

    The first: heap overflow.

    This exception message will be thrown in the JVM if 98% of the time is used for GC and the available Heap size is less than 2%.

    If there is no memory leak, adjusting the -Xms -Xmx parameters can solve it.

    -Xms: initial heap size

    -Xmx: maximum heap size

    but the size of the heap is affected by the following three aspects:

    1. The data model (32-bt or 64-bit) of the relevant operating system is limited; (under the 32-bit system, it is generally limited to 1.5G~2G; I am under the 2003 server system (physical memory: 4G and 6G, jdk: 1.6 ) Test 1612M, 64 is that the operating system has unlimited memory.)

    2. The system's available virtual memory limit;

    3. The system's available physical memory limit.

    The size of the heap can be tested using the java -Xmx***M version command. If it is supported, the version number of jdk will appear. If it is not supported, an error will be reported.

    -Xms -Xmx are generally configured to be the same. For example, set JAVA_OPTS= -Xms1024m -Xmx1024m

    The second type: permanent storage area overflow

    The full name of PermGen space is Permanent Generation space, which refers to the permanent storage area of ​​memory. This part is used to store the information of Class and Meta. Class is put into the PermGen space area when it is loaded. It is different from the Heap area where Instance is stored. GC (Garbage Collection) will not perform PermGen space operations during the running of the main program. Clean up, so if your APP loads a lot of CLASS, it is likely to have a PermGen space error. This kind of error is common when the web server precompiles the JSP. But the current hibernate and spring projects are also prone to such problems. http://www.iteeye.com/topic/80620? The post on page=1 discusses this issue. It may be because these frameworks will dynamically class, and the gc of the jvm will not clean up the PemGen space, resulting in memory overflow.

    This one is generally to increase -XX:PermSize -XX:MaxPermSize to solve the problem.

    -XX: PermSize permanent storage area initial size

    -XX: PermSize permanent storage area initial maximum value

    This is generally used in conjunction with the first item, such as set JAVA_OPTS= -Xms1024m -Xmx1024m -XX: PermSize=128M -XX: PermSize=256M

    has a little need Note: The maximum heap memory tested by the java -Xmx***M version command is the sum of -Xmx and -XX: PermSize. For example, the system supports the largest jvm heap size of 1.5G, then -Xmx1024m -XX: PermSize=768M is impossible In progress.

    Third: Unable to create a new thread.

    This phenomenon is relatively rare and strange, mainly related to the ratio of jvm to system memory.

    This weirdness is because the JVM has been allocated a lot of memory by the system (say 1.5G) and it takes at least half of the available memory. It has been found that with a large number of threads, the more memory you allocate to the JVM, the more likely the above error will occur.

    The reasons for this phenomenon are as follows (learn the reasons from this blog: http://hi.baidu.com/hexiong/blog/item/16dc9e518fb10c2542a75b3c.html):

    Each 32-bit process can use up to 2G of available memory, because the other 2G is reserved by the operating system. It is assumed here that 1.5G is used for the JVM, so there is still 500M of available memory. Part of this 500M memory must be used for system dll loading, so the real remaining may be only 400M. Now the key point appears: when you use Java to create a thread, a Thread object will also be created in the JVM memory, But at the same time, a real physical thread will be created in the operating system (refer to the JVM specification). In jdk1.4, the default stack size is 256KB, but in jdk1.5, the default stack size is 1M per thread, so we can only create up to 400 available threads in the remaining 400M of available memory.

    This concludes that to create more threads, you must reduce the maximum memory allocated to the JVM. Another option is to have the JVM host inside your JNI code.

    Give an estimation formula about the maximum number of threads that can be created:

    (MaxProcessMemory - JVMMemory - ReservedOsMemory) / (ThreadStackSize) = Number of threads

    For jdk1.5, assuming that the operating system reserves 120M memory:

    1.5GB JVM: (2GB -1.5Gb-120MB)/(1MB) = ~380 threads

    1.0GB JVM: (2GB-1.0Gb-120MB)/(1MB) = ~880 threads

    There is a startup option in the boot.ini of 2000/XP/2003, It seems to be: /PAE /3G, which allows the user process to expand the maximum memory to 3G, and the operating system can only occupy a maximum of 1G of virtual memory. That should allow the JVM to create more threads.

    Therefore, this situation needs to be adjusted in conjunction with the operating system.

    Therefore: We need to perform different diagnosis of tomcat memory allocation in combination with different situations to fundamentally solve the problem