Insufficient JVM Code Cache space, resulting in slow service performance
https://www.jianshu.com/p/4214df5df334
According to business feedback, after an online application has been running for a period of time, after the peak period, I suddenly found that the processing capacity has decreased and the response time of the interface has become longer, but looking at the GC data on Cat, everything is normal.
Check the logs through the machine on the springboard and find a log that is rarely seen:
Java HotSpot(TM) 64-Bit Server VM warning: CodeCache is full. Compiler has been disabled. Java HotSpot(TM) 64-Bit Server VM warning: Try increasing the code cache size using -XX:ReservedCodeCacheSize=. ... “CompilerThread0” java.lang.OutOfMemoryError: requested 2854248 bytes for Chunk::new. Out of swap space? CodeCache is full, indicating that Code Cache is full, which causes Compiler to fail. Why is this?
First, we have to understand what is Code Cache.
What is Code Cache
Java code will trigger JIT compilation when the number of executions reaches a threshold. Once the code block is compiled into local machine code, the compiled local machine code will be run directly the next time it is executed. Therefore, the local machine code must be cached, and the memory area of the cached local machine code is Code Cache, which is not part of the Java heap. In addition to the JIT compiled code, the native method code (JNI) used by Java It also exists in codeCache.
Code Cache tuning
Since Code Cache is a memory area, there must be a size limit, but for different versions of JVM and different startup methods, the default size of Code Cache is also different, which can jinfo -flag ReservedCodeCacheSize be checked.
After the service is started, as time goes by, there will definitely be more and more methods compiled by JIT into local machine code and stored in Code Cache. Since the size of Code Cache is fixed, there is a risk of being used up.
Once the Code Cache is filled, the following will occur:
- The JIT function of the JVM will be stopped and no additional code will be compiled.
- The compiled code is still executed in a compiled manner, but the code that has not been compiled can only be executed in an interpreted manner.
In this case, if there is still a lot of code in the application executed in an interpreted manner, its performance will be greatly reduced. In order to avoid this situation, you need a deeper understanding of Code Cache.
When the JVM starts, the memory required by Code Cache will be initialized separately. At this time, the Java heap will also be initialized, so Code Cache and Java heap are two separate memory areas.
In codeCache.cppthe CodeCache::initialize()method, the initialization of Code Cache is realized
Code Cache contains 3 types of data:
- NonNMethodCode
- ProfiledCode
- NonProfiledCode
Through the SegmentedCodeCacheparameters, you can choose whether to initialize according to the whole or segment.
-XX:ReservedCodeCacheSizeThe initialization size of Code Cache can be specified through parameters. This default value is different in different JDK versions. At present, I am debugging OpenJDK11 here. The default size is 240M, which is enough.
You can see the default size of other versions:
For those with only 32M and 48M, there may be a hidden danger of insufficient Code Cache, and the increase ReservedCodeCacheSizemay be a solution, but this is usually only a temporary solution.
Fortunately, the JVM provides a more radical codeCache recovery method: Speculative flushing.
This recovery method is enabled by default after JDK1.7.0_4, and the previous version needs to be enabled with a parameter: -XX: + UseCodeCacheFlushing.
With Speculative flushing enabled, when Code Cache is insufficient:
- The earliest half of the compiled methods will be put in an old list for recycling;
- Within a certain time interval, if the method in the old list is not called, this method will be cleared from the Code Cache;
Unfortunately, in JDK1.7, Speculative flushing freed up some space, but from the compilation log, JIT did not return to normal, and the overall performance of the system dropped a lot, and a large number of timeouts occurred.
I saw such a bug on Oracle's official website: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8006952 Due to algorithm problems, when the Code Cache is insufficient, the compilation thread cannot continue and consume a lot of CPU. Causes the system to run slower.
