Linux system CPU occupancy rate is higher troubleshooting ideas

As Linux operation and maintenance engineers, in their daily work situations we encounter high CPU load of 100% appears on the Linux server, if the CPU continued to run high, it will affect the normal operation of business systems, bringing business losses.

Operation and maintenance of many students encounter this situation often at a loss for CPU overload usually used to quickly locate two ways:

method one

*** Step Three: Use

1. top, and then press shift + p sorted according CPU 

Find high CPU-intensive process of pid

Step 2: Use

1. top -H -p [process id] 

Find the process of resource consumption *** thread id

The third step: Use

1. echo ' the obase = 16; [thread id]' | bc or printf "% x \ n" [thread id]  

The conversion thread id hexadecimal (lowercase letters)

1. bc is a linux command calculator 

Step four: Perform

1. jstack [process id] | grep -A 10 [hex thread id] " 

Viewing thread status information

Method Two

*** Step Three: Use

1. top, and then press shift + p sorted according CPU 

Find high CPU-intensive processes

Step 2: Use

1. ps -mp pid -o THREAD,tid,time | sort -rn 

Get thread information, and to find high CPU-intensive thread

The third step: Use

1. echo ' the obase = 16; [thread id]' | bc or printf "% x \ n" [thread id] 

The conversion requires a thread ID in hex format

Step Four: Use

1. jstack pid | grep tid -A 30 [id thread hex] 

Print thread stack information

case study

1. Scene Description

JAVA process troubleshooting high CPU utilization in a production environment

2. settlement process

(1) According to the top order, find the PID of the Java process CPU-2633 up to 300% failure.

(2) find the process, how to locate specific thread or code it, first of all display the thread list and sorted according to high CPU utilization threads:

1. [root@localhost ~]# ps -mp 2633 -o THREAD,tid,time | sort -rn 

The results show the following:

Find the time-consuming *** thread (TID) 3626, CPU time is 12 minutes!

(3) the need to thread TID converted to hexadecimal format

1. [root@localhost ~]# printf "%x\n" 3626 
2. E18 

(4) *** Use jstack command to print out the stack information of the process following this thread:

1. [root@localhost ~]# jstack 2633 |grep "e18" -A 30 

For failure to solve compared and found that failure is equally important to the majority of the monitoring software on the market can achieve real-time observation server load, such as:! Zabbix, Nagios, Ali cloud monitor (for the cloud server) and so on. But among most of the software requires students to take the initiative to set up the operation and maintenance rules or testing to find the problem, how can passively receive alerts it?