Java command learning series (two)-Jstack
Included in the topic
#
7 Linux commands
jstack is a stack trace tool that comes with the java virtual machine.
Features
jstack is used to generate a thread snapshot of the java virtual machine at the current moment. Thread snapshot is a collection of method stacks being executed by each thread in the current java virtual machine. The main purpose of generating thread snapshots is to locate the cause of long pauses in threads, such as deadlocks between threads, endless loops, and long periods caused by external resource requests. Time to wait and wait. When a thread is stalled, you can view the call stack of each thread through jstack, and you can know what the unresponsive thread is doing in the background, or what resources it is waiting for. If the java program crashes to generate a core file, the jstack tool can be used to obtain the java stack and native stack information of the core file, so that you can easily know how the java program crashed and where the problem occurred. In addition, the jstack tool can also be attached to the running java program, and you can see the java stack and native stack information of the running java program at the time. If the currently running java program shows a hung state, jstack is very useful.
The So,jstack command is mainly used to view the call stack of the Java thread and can be used to analyze thread problems (such as deadlock).
Thread state
If you want to analyze the situation of threads through the jstack command, you must first know what state the thread has. The following states are the several states of the thread that we may see when we use the jstack command to view thread stack information:
NEW, not activated. Will not appear in Dump.
RUNNABLE, executed in the virtual machine.
BLOCKED, blocked and waiting for the monitor to lock.
WATING, waiting indefinitely for another thread to perform a specific operation.
TIMED_WATING, time-limited waiting for a specific operation of another thread.
TERMINATED, has been withdrawn.
Monitor
In a multi-threaded JAVA program, to achieve synchronization between threads, it is necessary to talk about Monitor. Monitor is the main method used to achieve mutual exclusion and cooperation between threads in Java. It can be seen as a lock of an object or a class. Every object has, and only one monitor. The following figure describes the relationship between the thread and the Monitor, and the state transition diagram of the thread:
Entrt Set: Indicates that the thread acquires the lock of the object through the synchronized request. If the object is not locked, enter the owner; otherwise, wait in the entry zone. Once the object lock is released by other threads, it immediately participates in the competition.
Owner (The Owner): Indicates that a thread successfully competed for the object lock.
Waiting area (Wait Set): Indicates that the thread releases the lock of the object through the wait method of the object, and waits in the waiting area to be awakened.
It can be seen from the figure that a Monitor can only be owned by one thread at a certain time. This thread is "Active Thread", and the other threads are "Waiting Thread", which are in two queues "Entry Set" and "Entry Set" and "Waiting Thread" respectively. Wait in "Wait Set". The state of the thread waiting in the "Entry Set" is "Waiting for monitor entry", and the state of the thread waiting in the "Wait Set" is "in Object.wait()". First look at the threads in the "Entry Set". We call the code segment protected by synchronized the critical section. When a thread applies to enter the critical section, it enters the "Entry Set" queue. The corresponding code is like:
synchronized(obj) {
.........
}Call decoration
Indicates the extra important operation of the thread when the method is called. Important information for thread Dump analysis. Decorate the method call above.
locked <address> Goal: successfully apply for object lock using synchronized, the owner of the monitor.
waiting to lock <address> Goal: Use synchronized to apply for object lock unsuccessfully, wait in the incoming area.
waiting on <address> Goal: After successfully applying for object lock using synchronized, release the lock and wait in the waiting area.
parking to wait for <address> target
locked
at oracle.jdbc.driver.PhysicalConnection.prepareStatement
- locked <0x00002aab63bf7f58> (a oracle.jdbc.driver.T4CConnection)
at oracle.jdbc.driver.PhysicalConnection.prepareStatement
- locked <0x00002aab63bf7f58> (a oracle.jdbc.driver.T4CConnection)
at com.jiuqi.dna.core.internal.db.datasource.PooledConnection.prepareStatementThrough the synchronized keyword, successfully obtained the lock of the object, became the owner of the monitor, and operated in the critical area. Object locks are thread-reentrant.
waiting to lock
at com.jiuqi.dna.core.impl.CacheHolder.isVisibleIn(CacheHolder.java:165)
- waiting to lock <0x0000000097ba9aa8> (a CacheHolder)
at com.jiuqi.dna.core.impl.CacheGroup$Index.findHolder
at com.jiuqi.dna.core.impl.ContextImpl.find
at com.jiuqi.dna.bap.basedata.common.util.BaseDataCenter.findInfoThrough the synchronized keyword, the lock of the object is not acquired, and the thread waits in the entry area of the monitor. Appears at the top of the call stack, and the thread status is Blocked.
waiting on
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000da2defb0> (a WorkingThread)
at com.jiuqi.dna.core.impl.WorkingManager.getWorkToDo
- locked <0x00000000da2defb0> (a WorkingThread)
at com.jiuqi.dna.core.impl.WorkingThread.runThrough the synchronized keyword, after successfully obtaining the lock of the object, call the wait method and enter the waiting area of the object to wait. At the top of the call stack, the thread status is WAITING or TIMED_WATING.
parking to wait for
park is a basic thread blocking primitive and does not block on objects through the monitor. The new mechanism that appears with the concurrent package is different from the unsynchronized system.
Thread action
The reason for the thread state
runnable: The status is generally RUNNABLE.
in Object.wait(): Waiting in the waiting area, the status is WAITING or TIMED_WAITING.
waiting for monitor entry: waiting for entering the zone, the state is BLOCKED.
waiting on condition: waiting in the waiting area, being parked.
sleeping: The sleeping thread has called Thread.sleep().
Wait on condition This state appears when the thread is waiting for the occurrence of a certain condition. The specific reason can be analyzed in combination with stacktrace. The most common situation is that the thread is in the sleep state, waiting to be awakened. A common situation is to wait for network IO: Before java introduced nio, for each network connection, there was a corresponding thread to handle network read and write operations. Even if there is no readable and writable data, the thread is still blocked in read and write operations. On the other hand, this may cause a waste of resources and also put pressure on the thread scheduling of the operating system. A new mechanism is adopted in NewIO, and the performance and scalability of the written server program have been improved. Waiting for the network to read and write, this may be a sign of a network bottleneck. The thread cannot be executed because of network congestion. One situation is that the network is very busy, almost all bandwidth is consumed, and there is still a large amount of data waiting for the network to read and write; the other situation may also be that the network is idle, but due to routing problems, packets cannot arrive normally. Therefore, it is necessary to combine some performance observation tools of the system for comprehensive analysis, such as netstat statistics of the number of sent packets per unit time, if it obviously exceeds the limit of the network bandwidth; observe the utilization of the cpu, if the CPU time in the system state, relative to The proportion of CPU time in the user mode is relatively high; if the program is running on the Solaris 10 platform, you can use the dtrace tool to see the system call situation, if you observe the number of read/write system calls or the running time is far ahead; these all point to the network Network bottlenecks caused by bandwidth limitations. (From http://www.blogjava.net/jzone/articles/303979.html)
Analysis of Thread Dump
in principle
Combine code reading reasoning. Requires mutual derivation and verification of thread dump and source code.
The root cause of the bug is often not directly reflected on the call stack, so you must pay special attention to all calls before the current call of the thread.
Start point
Waiting in the area
"d&a-3588" daemon waiting for monitor entry [0x000000006e5d5000]
java.lang.Thread.State: BLOCKED (on object monitor)
at com.jiuqi.dna.bap.authority.service.UserService$LoginHandler.handle()
- waiting to lock <0x0000000602f38e90> (a java.lang.Object)
at com.jiuqi.dna.bap.authority.service.UserService$LoginHandler.handle()Thread state BLOCKED, thread action wait on monitor entry, call modification waiting to lock always appear together. Indicates that there is already a conflicting call at the code level. There must be a problematic code, and its occurrence needs to be minimized.
Sync block blocking
One thread locks an object, and a large number of other threads wait on the object.
"blocker" runnable
java.lang.Thread.State: RUNNABLE
at com.jiuqi.hcl.javadump.Blocker$1.run(Blocker.java:23)
- locked <0x00000000eb8eff68> (a java.lang.Object)
"blockee-11" waiting for monitor entry
java.lang.Thread.State: BLOCKED (on object monitor)
at com.jiuqi.hcl.javadump.Blocker$2.run(Blocker.java:41)
- waiting to lock <0x00000000eb8eff68> (a java.lang.Object)
"blockee-86" waiting for monitor entry
java.lang.Thread.State: BLOCKED (on object monitor)
at com.jiuqi.hcl.javadump.Blocker$2.run(Blocker.java:41)
- waiting to lock <0x00000000eb8eff68> (a java.lang.Object)Continuously running IO IO operations can be blocked in the RUNNABLE state. For example: database deadlock, network read and write. Pay special attention to the analysis of the true state of the IO thread. Generally speaking, IO calls that are caught in RUNNABLE are all problematic.
The following stack shows: The thread status is RUNNABLE. The call stack is on SocketInputStream or SocketImpl, socketRead0 and other methods. The call stack contains jdbc related packages. It is possible that a database deadlock has occurred
"d&a-614" daemon prio=6 tid=0x0000000022f1f000 nid=0x37c8 runnable
[0x0000000027cbd000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at oracle.net.ns.Packet.receive(Packet.java:240)
at oracle.net.ns.DataPacket.receive(DataPacket.java:92)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:172)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:117)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1034)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:588)Sleep with threaded scheduling
Normal thread pool waiting
"d&a-131" in Object.wait()
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at com.jiuqi.dna.core.impl.WorkingManager.getWorkToDo(WorkingManager.java:322)
- locked <0x0000000313f656f8> (a com.jiuqi.dna.core.impl.WorkingThread)
at com.jiuqi.dna.core.impl.WorkingThread.run(WorkingThread.java:40)Suspicious thread waiting
"d&a-121" in Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.jiuqi.dna.core.impl.AcquirableAccessor.exclusive()
- locked <0x00000003011678d8> (a com.jiuqi.dna.core.impl.CacheGroup)
at com.jiuqi.dna.core.impl.Transaction.lock()Start point summary
wait on monitor entry: blocked, there must be a problem
runnable: pay attention to the IO thread
in Object.wait(): Pay attention to non-thread pool waiting
use
If you want to learn a command, first look at the help. Use jstack -help to view the help:
hollis@hos:~$ jstack -help
Usage:
jstack [-l] <pid>
(to connect to running process)
jstack -F [-m] [-l] <pid>
(to connect to a hung process)
jstack [-m] [-l] <executable> <core>
(to connect to a core file)
jstack [-m] [-l] [server_id@]<remote server IP or hostname>
(to connect to a remote debug server)
Options:
-F to force a thread dump. Use when jstack <pid> does not respond (process is hung)
-m to print both java and native frames (mixed mode)
-l long listing. Prints additional information about locks
-h or -help to print this help message-F Force to print stack information when there is no corresponding'jstack [-l] pid' -l long list. Print additional information about locks, such as the list of ownable synchronizers belonging to java.util.concurrent. -m print java and native c /c++ framework all stack information.-h | -help Print help information pid The id of the java process that needs to be printed configuration information, you can use jps to query.
First, we analyze the thread situation of such a program:
/**
* @author hollis
*/
public class JStackDemo1 {
public static void main(String[] args) {
while (true) {
//Do Nothing
}
}
}First, there is jps to view the process number:
hollis@hos:~$ jps
29788 JStackDemo1
29834 Jps
22385 org.eclipse.equinox.launcher_1.3.0.v20130327-1440.jarThen use jstack to view stack information:
hollis@hos:~$ jstack 29788
2015-04-17 23:47:31
...此处省略若干内容...
"main" prio=10 tid=0x00007f197800a000 nid=0x7462 runnable [0x00007f197f7e1000]
java.lang.Thread.State: RUNNABLE
at javaCommand.JStackDemo1.main(JStackDemo1.java:7)What can we see from this stack information? We can see that there is currently a user-level thread, which is in the runnable state and executes to the seventh line of JStackDemo1.java. Look at the following code:
/**
* @author hollis
*/
public class JStackDemo1 {
public static void main(String[] args) {
Thread thread = new Thread(new Thread1());
thread.start();
}
}
class Thread1 implements Runnable{
@Override
public void run() {
while(true){
System.out.println(1);
}
}
}The thread stack information is as follows:
"Reference Handler" daemon prio=10 tid=0x00007fbbcc06e000 nid=0x286c in Object.wait() [0x00007fbbc8dfc000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000783e066e0> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked <0x0000000783e066e0> (a java.lang.ref.Reference$Lock)We can see:
Thread status: WAITING thread's call stack thread's currently locked resources: <0x0000000783e066e0> thread currently waiting for resources: <0x0000000783e066e0>
Why are waiting for the same resource locked at the same time:
In the execution of the thread, the Monitor of this object is obtained first (corresponding to locked <0x0000000783e066e0>). When the execution reaches obj.wait(), the thread gives up the ownership of the Monitor and enters the "wait set" queue (corresponding to waiting on <0x0000000783e066e0>).
Deadlock analysis
After learning how to use the jstack command, we can see how to use jstack to analyze deadlocks. This is what we must master. What is a deadlock? The so-called deadlock: refers to a blocking phenomenon caused by competition for resources or due to communication with each other during the execution of two or more processes. If there is no external force, they will not be able to advance. At this time, the system is said to be in a deadlock state or the system has a deadlock. These processes that are always waiting for each other are called deadlock processes. To put it bluntly, I want to eat egg-filled pancakes. There are eggs and pancakes on the table, but my friend and I picked up the eggs and the sickness at the same time. I have eggs in my hand, but I need the pancakes in his hands. He has bread in his hand, but he wants the egg in my hand. In this way, if we can't get the eggs and the pie at the same time, then we can't continue to do the following work (making egg filling pie). Therefore, this caused a deadlock. Look at a deadlocked program:
package javaCommand;
/**
* @author hollis
*/
public class JStackDemo {
public static void main(String[] args) {
Thread t1 = new Thread(new DeadLockclass(true));//建立一个线程
Thread t2 = new Thread(new DeadLockclass(false));//建立另一个线程
t1.start();//启动一个线程
t2.start();//启动另一个线程
}
}
class DeadLockclass implements Runnable {
public boolean falg;// 控制线程
DeadLockclass(boolean falg) {
this.falg = falg;
}
public void run() {
/**
* 如果falg的值为true则调用t1线程
*/
if (falg) {
while (true) {
synchronized (Suo.o1) {
System.out.println("o1 " + Thread.currentThread().getName());
synchronized (Suo.o2) {
System.out.println("o2 " + Thread.currentThread().getName());
}
}
}
}
/**
* 如果falg的值为false则调用t2线程
*/
else {
while (true) {
synchronized (Suo.o2) {
System.out.println("o2 " + Thread.currentThread().getName());
synchronized (Suo.o1) {
System.out.println("o1 " + Thread.currentThread().getName());
}
}
}
}
}
}
class Suo {
static Object o1 = new Object();
static Object o2 = new Object();
}When I start the program, we look at the console:
We found that the program only output two lines, and then the program no longer prints other things, but the program did not stop. This creates a deadlock. When thread 1 uses synchronized to lock o1, thread 2 also uses synchronized to lock o2. When both threads finish the first print task, thread 1 wants to lock o2, and thread 2 wants to lock o1. However, thread 1 currently locks o1 and thread 2 locks o2. Therefore, the two think that Chengdu cannot continue the execution, which caused a deadlock.
Then, we use jstack to look at the thread stack information:
Found one Java-level deadlock:
=============================
"Thread-1":
waiting to lock monitor 0x00007f0134003ae8 (object 0x00000007d6aa2c98, a java.lang.Object),
which is held by "Thread-0"
"Thread-0":
waiting to lock monitor 0x00007f0134006168 (object 0x00000007d6aa2ca8, a java.lang.Object),
which is held by "Thread-1"
Java stack information for the threads listed above:
===================================================
"Thread-1":
at javaCommand.DeadLockclass.run(JStackDemo.java:40)
- waiting to lock <0x00000007d6aa2c98> (a java.lang.Object)
- locked <0x00000007d6aa2ca8> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:745)
"Thread-0":
at javaCommand.DeadLockclass.run(JStackDemo.java:27)
- waiting to lock <0x00000007d6aa2ca8> (a java.lang.Object)
- locked <0x00000007d6aa2c98> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:745)
Found 1 deadlock.Haha, the stack is clearly written, it tells us Found one Java-level deadlock, and then points out the contents of the two threads that caused the deadlock. Then, through the Java stack information for the threads listed above to display more detailed deadlock information. He said
When Thread-1 wants to execute line 40, it currently locks the resource <0x00000007d6aa2ca8>, but it is waiting for the resource <0x00000007d6aa2c98>. When Thread-0 wants to execute line 27, it currently locks the resource <0x00000007d6aa2c98 >, but he is waiting for resources <0x00000007d6aa2ca8> Because these two threads both hold resources and both need resources from each other, a deadlock is caused. If we find the reason, we can analyze the specific problem and solve the deadlock.
other
When the virtual machine executes Full GC, it will block all user threads. Therefore, the thread that immediately acquires the synchronization lock may also be blocked. When viewing the thread dump, first look at the memory usage.