Recently looked at the background of the game server deployment status, I found that when I grow more than 100% of a Java program occupied by CPU, the investigation found was actually caused by Disruptor, let's look at Disruptor Why is there such a performance.
Found that more than 100 percent CPU time of the process
First see the status with the server on the top order, found 100% longer than the time occupied by the application has a CPU, a shown in FIG:
I checked according to the process numbers a bit and found my game a Java background service, there is a CPU is almost full, and therefore continue to troubleshoot what is the code that led to this situation.
Use top -Hp 27538will be displayed all the threads of this process, in accordance with the CPU utilization time ordering, see this result:
27658 thread takes up almost all of the CPU time, and always has been, so to view details of this process.
With jstack pid > pid.logan order for the process to a process of the snapshot output file downloaded.
27658 will be converted to hexadecimal 0x6c0a query (the snapshot because the thread hex thread ID is stored, so need to be converted) in the thread snapshot:
"disruptor-0" #27 prio=5 os_prio=0 tid=0x00007fa100c58000 nid=0x6c0a runnable [0x00007fa0ae080000]
java.lang.Thread.State: RUNNABLE
at com.lmax.disruptor.BusySpinWaitStrategy.waitFor(BusySpinWaitStrategy.java:39)
at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
at com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:159)
at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125)
at java.lang.Thread.run(Thread.java:748)This is a stack Disruptor, in order to better visualize thread status information can be uploaded to a dedicated snapshot of the analysis platform .
(Blogger himself for a snapshot analysis is also in the process of the novice stage, if you have any suggestions or comments, please leave a comment below.)
Why Disruptor analysis will take up the entire CPU
According to the analysis above snapshot, Disruptor is actually holding policy related to the thread caused to view the BusySpinWaitStrategyclass and found instructions:
* This strategy will use CPU resource to avoid syscalls which can introduce latency jitter. It is best
* used when threads can be bound to specific CPU cores.Now we finally know, the original is because of this strategy is to make a binding thread CPU core, its natural CPU usage more than 100% of the time.
to sum up
By this time troubleshooting the problem, not only to understand the linux system processes, threads of relationships, also started investigation java online service, by the way also reviewed what had previously Disruptor contact. If you have any suggestions or comments, please leave a comment below.
Are interested can follow my public number, maybe there will be surprises.
