Morning receive a text message, specific alarm message is as follows:
[UMP JVM memory alarm monitoring application name]: a ticket issuing worker (jdos_couponwkr); KEY] [coupon.send.worker.jvm, hostname: host-10-183-72-114 [], [11909223645] Examples of heap memory usage 3 times exceeds a set threshold [90.0%]. Alarm level: [Warning], the alarm time: [2019-07-17 07:36:12].
He said that there is a machine heap memory usage exceeds the threshold of 90%. Although this message concise, but hidden behind the technical details, I am here to state what, fallacious, please advise.
At first glance this alarm, then know heap memory usage exceeds a threshold value. The first reaction is usually within the stack memory transfer large, on the line. But this really true? Let's take a look.
Suppose we are now in the heap memory is 1G, the heap memory usage curve as follows:
You can see the overall normal use, recycling normal, very smooth.
Now, we have an in-memory heap adjusted to 2G, within the heap memory usage curve as follows:
It can be seen in the heap memory usage still exceeds 90%, indicating a simple adjustment in the heap memory size, you can not solve this problem.
why? Because when jvm work in the use of heap memory is adaptive, you give more, it uses much less you give, it uses less. So this is why you give it a 1G size, it will be more than 90% of the memory heap used, you give him a 2G size, he can stack more than 90% of memory used. Of course, the premise is to use a large amount of memory the application for the job in 2G. Otherwise assigned to 2G heap memory is not used so high. 1.5G or so may be used on the release.
Come talk about the garbage collection mechanism. Here I G1 collectively outside the old algorithm for the algorithm. Let's compare:
In the heap memory Docker 8G, the same application, the same operations, the pressure test results are as follows:
1. young gc times, 20 times the old algorithm, G1 algorithm 8 times.
2. The maximum heap memory usage, the old algorithm uses the largest 6.3G, G1 algorithm uses the largest 7.6G.
As can be seen, in the heap memory 8G scene, excellent overall performance algorithm G1, gc fewer times, more efficient utilization of heap memory.
But 8G heap memory used in the following scenarios, G1 algorithm the advantage is not obvious. It is strongly recommended that larger application heap memory to open G1 algorithm.
Finally said, encounter this prompt message content needs to be checked.
1. Check yong gc time-consuming, consuming an average of 40ms or less normal, more than 1s you need to troubleshoot the problem.
2. Check the full number of gc, several times or dozens of times within a minute it appears that frequent need to troubleshoot.
3. Check the graphics heap memory recovery, if memory usage go up, but the delay get down, it means years old can not be recycled, check the code to identify the reasons, a figure similar to the following:
Can be seen in this figure, the curve up to get down, because the application of internal local cache has been running and no mechanisms leading to expire, and later closed out this local cache is recycled properly.
The highest point of the curve to see if usage jvm heap memory size in line with the current setting, if the current heap memory usage has been higher vertex, the application needs more memory than the heap memory allocated to a large number, may be enough memory resources Some try to allocate more than in the case of heap memory.