Java online application troubleshooting one: high CPU usage [transfer]

http://blog.csdn.net/blade2001/article/details/9065985

An application consumes a lot of CPU, except that it is indeed a computing-intensive application, usually the reason is an infinite loop.

(Friendly reminder: This blog article is welcome to reprint, but please indicate the source: hankchen, http://www.blogjava.net/hankchen)

Taking an actual fault that occurred recently as an example, we will introduce how to locate and solve such problems.

clip_image002

According to the top command, it is found that the Java process with PID 28555 occupies up to 200% of the CPU and is faulty.

Through the ps aux | grep PID command, it can be further determined that there is a problem with the tomcat process. However, how to locate a specific thread or code?

First display the thread list:

ps -mp pid -o THREAD,tid,time

1

Find the most time-consuming thread 28802, occupying CPU time for almost two hours!

Second, convert the required thread ID to hexadecimal format:

printf "%x\n" tid

2

Finally, print the stack information of the thread:

jstack pid |grep tid -A 30

3

Find the code with the problem!

Now let's analyze the specific code: ShortSocketIO.readBytes(ShortSocketIO.java:106)

ShortSocketIO是应用封装的一个用短连接Socket通信的工具类。readBytes函数的代码如下：

public byte[] readBytes(int length) throws IOException {

    if ((this.socket == null) || (!this.socket.isConnected())) {

        throw new IOException("++++ attempting to read from closed socket");

    }

    byte[] result = null;

    ByteArrayOutputStream bos = new ByteArrayOutputStream();

    if (this.recIndex >= length) {

           bos.write(this.recBuf, 0, length);

           byte[] newBuf = new byte[this.recBufSize];

           if (this.recIndex > length) {

               System.arraycopy(this.recBuf, length, newBuf, 0, this.recIndex - length);

           }

           this.recBuf = newBuf;

           this.recIndex -= length;

    } else {

           int totalread = length;

           if (this.recIndex > 0) {

                totalread -= this.recIndex;

                bos.write(this.recBuf, 0, this.recIndex);

                this.recBuf = new byte[this.recBufSize];

                this.recIndex = 0;

    }

    int readCount = 0;

    while (totalread > 0) {

         if ((readCount = this.in.read(this.recBuf)) > 0) {

                if (totalread > readCount) {

                      bos.write(this.recBuf, 0, readCount);

                      this.recBuf = new byte[this.recBufSize];

                      this.recIndex = 0;

               } else {

                     bos.write(this.recBuf, 0, totalread);

                     byte[] newBuf = new byte[this.recBufSize];

                     System.arraycopy(this.recBuf, totalread, newBuf, 0, readCount - totalread);

                     this.recBuf = newBuf;

                     this.recIndex = (readCount - totalread);

             }

             totalread -= readCount;

        }

   }

}

The problem lies in the code marked in red. If the data returned by this.in.read() is less than or equal to 0, the loop continues. This can happen when the network is congested.

As for how to modify it, it depends on how the business logic should treat this special situation.



Finally, summarize the methods and techniques for troubleshooting CPU failures:

1. top command: Linux command. You can view real-time CPU usage. You can also view the CPU usage of the recent period.

2. PS command: Linux command. Powerful process status monitoring commands. You can view the current CPU usage of the process and the threads in the process. Sampled data belonging to the current state.

3. jstack: Command provided by Java. You can view the current thread stack operation of a process. According to the output of this command, you can locate the current running status of all threads of a process, running code, and whether it is deadlocked and so on.

4, pstack: Linux command. You can view the current thread stack operation of a process.