- JVM监控和诊断概述
程序运行中经常会遇到各种问题，定位问题时通常需要综合各种信息，如系统日志、堆dump文件、线程dump文件、GC日志等。通过虚拟机监控和诊断工具可以帮忙我们快速获取、分析需要的数据，进而提高问题解决速度。可以使用图形化工具如:jconsole,jvisualvm,或者命令进行监控和诊断命令。 本文将介绍虚拟机常用监控和问题诊断命令工具的使用方法，主要包含以下工具:
| jps | 显示系统中所有Hotspot虚拟机进程 |
| jstat | 收集Hotspot虚拟机各方面运行数据 |
| jstack | 显示虚拟机的线程栈信息 |
| jinfo | 显示虚拟机的配置信息 |
| jmap | 用于生成虚拟机的内存快照信息 |
- 命令介绍
1).jps 命令
JVM Process Status Tool，该命令用于列出正在运行的虚拟机进程，显示main类的名称和虚拟机进程id。该命令受当前用户的访问权限影响，比如linux下非root用户只列出当前用户启动的虚拟机进程。
命令格式:
jps [options] [hostid]
执行示例:
glassfish@0220:~$ jps -l 32214 sun.tools.jps.Jps 11850 com.sun.enterprise.glassfish.bootstrap.ASMain
常用参数:
| -l | 输出主类全名 |
| -v | 输出虚拟机进程启动的jvm参数 |
| -m | 输出启动时传递给main函数的参数 |
2).jstack 命令
Stack Trace for Java，用于生成虚拟机当前的线程快照信息，包含每一条线程的堆栈信息。该命令通常用于定位线程停顿原因，当出现线程停顿时，可通过stack查看每个线程的堆栈信息，进而分析停顿原因。
命令格式:
jstack [ option ] pid
常用参数:
| -l | 除堆栈外，显示锁的附加信息 |
| -F | 当请求不被响应时，强制输出线程堆栈 |
| -m | 混合模式，打印java和本地C++调用的堆栈信息 |
执行示例：
"Thread-11" #24 daemon prio=5 os_prio=0 tid=0x0000000001b55800 nid=0x2e67 waiting on condition [0x00007ff666bfc000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e1b8ccb8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
"Thread-9" #21 daemon prio=5 os_prio=0 tid=0x0000000001b0a000 nid=0x2e64 waiting on condition [0x00007ff66c346000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e1e1a310> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
at com.sun.common.util.logging.LoggingOutputStream.log(LoggingOutputStream.java:144)
at com.sun.common.util.logging.LoggingOutputStream$1.run(LoggingOutputStream.java:123)
"Thread-8" #20 daemon prio=5 os_prio=0 tid=0x0000000001b08800 nid=0x2e63 waiting on condition [0x00007ff66c447000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e1e0b4e8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
at com.sun.common.util.logging.LoggingOutputStream.log(LoggingOutputStream.java:144)
at com.sun.common.util.logging.LoggingOutputStream$1.run(LoggingOutputStream.java:123)
"pool-1-thread-1" #18 daemon prio=5 os_prio=0 tid=0x0000000001af4800 nid=0x2e61 waiting on condition [0x00007ff66c649000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e1a12528> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
"Timer-0" #13 daemon prio=5 os_prio=0 tid=0x00007ff6844f7000 nid=0x2e5c in Object.wait() [0x00007ff66cdf8000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e15087b0> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:502)
at java.util.TimerThread.mainLoop(Timer.java:526)
- locked <0x00000000e15087b0> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)
"FelixDispatchQueue" #11 daemon prio=5 os_prio=0 tid=0x00007ff64400e800 nid=0x2e5a in Object.wait() [0x00007ff66e36a000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:1063)
- locked <0x00000000e0c8da80> (a java.util.ArrayList)
at org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:54)
at org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:101)
at java.lang.Thread.run(Thread.java:745)
"Service Thread" #7 daemon prio=9 os_prio=0 tid=0x00007ff6841ef000 nid=0x2e57 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007ff6841d2000 nid=0x2e56 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007ff6841d0000 nid=0x2e55 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007ff684135800 nid=0x2e54 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007ff684105000 nid=0x2e53 in Object.wait() [0x00007ff66eff6000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142)
- locked <0x00000000e0c753b8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007ff684103000 nid=0x2e52 in Object.wait() [0x00007ff66f0f7000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
- locked <0x00000000e048ac30> (a java.lang.ref.Reference$Lock)
"VM Thread" os_prio=0 tid=0x00007ff6840fc000 nid=0x2e51 runnable
"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007ff684023000 nid=0x2e4f runnable
"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007ff684025000 nid=0x2e50 runnable
"VM Periodic Task Thread" os_prio=0 tid=0x00007ff6841f1800 nid=0x2e58 waiting on condition
JNI global references: 298
3) jstat命令
JVM Statistics Monitoring Tool，用于监控各种运行状态信息的命令。在只有文本控制台的环境中(如企业中的生产环境)，该工具非常有用。 可以用来显示系统中类装载、垃圾回收、运行期编译状况等运行数据。
命令格式:
jstat [ generalOption | outputOptions vmid [interval[s|ms] [count]] ]
vmid表示虚拟机唯一标识符，如果是本地虚拟机进程，与LVMID一致，通常为本地虚拟机进程号。 interval表示查询间隔时间，count表示查询次数。如果省略interval和count参数，表示查询一次。
常用参数:
| class | 类装载相关信息. |
| compiler | JIT编译器编译过的方法、耗时等. |
| gc | java堆信息和垃圾回收状况. |
| gccapacity | 关注java堆各个区的最大和最小空间. |
| gccause | 类似gcutil，额外输出导致上一次gc的原因. |
| gcnew | 新生代gc状况. |
| gcnewcapacity | 关注新生代gc使用的最大和最小空间. |
| gcold | 老年代gc状况. |
| gcoldcapacity | 关注老年代gc使用的最大和最小空间. |
| gcpermcapacity | 关注持久代gc使用的最大和最小空间. |
| gcutil | 关注已使用空间占总空间比例. |
| printcompilation | 输出已经被JIT编译的方法. |
执行示例
glassfish@0220:~$ jstat -gcutil 11850 1000 3 S0 S1 E O M CCS YGC YGCT FGC FGCT GCT 0.00 66.10 96.76 84.62 86.22 77.39 6404 119.294 158 103.270 222.564 0.00 66.10 96.76 84.62 86.22 77.39 6404 119.294 158 103.270 222.564 0.00 66.10 97.15 84.62 86.22 77.39 6404 119.294 158 103.270 222.564
S0和S1表示Survivor0和Survivor1，E表示新生代Eden，O表示老年代Old,M表示元空间大小，以上各参数值表示已使用空间占比。ccs压缩类空间， YGC表示young gc次数，YGCT表示young gc总耗时。FGC表示Full gc次数，FGCT表示full gc总耗时。GCT表示所有gc总耗时时间。jdk8以前： P表示持久代Permanent
4) jinfo命令
Configuration Info for Java，用于查看和修改虚拟机的各项参数信息。
命令格式：
jinfo [ option ] pid
常用参数:
| -flag name | 打印虚拟机该参数对应的值. |
| -flag [+\-]name | 使该参数生效或失效. |
| -flag name=value | 修改相应参数的值. |
| -flags | 打印传给jvm的参数值. |
| -sysprops | 打印System.getProperties()信息. |
5) jmap命令
Memory Map for Java，可以产生堆dump文件，查询堆和持久代的详细信息等。
命令格式:
jmap [ option ] pid
常用参数:
| -dump | 生成堆dump文件，格式为: -dump:[live,]format=b,file=<filename> |
| -heap | 显示java堆的详细信息，包括垃圾回收期、堆配置和分代信息等 |
| -histo | 显示堆中对象的统计信息，包括类名称，对应的实例数量和总容量 |
| -permstat | 统计持久代中各ClassLoader的统计信息。 |
示例：
jmap -dump:format=b,file=dump.tmp 11850 Dumping heap to /home/glassfish/dump.tmp ... Heap dump file created上面这个命令生成了dump.tmp这个dump文件。生成的dump文件可以使用Eclipse Memory Analyzer/jhat等工具进行分析
6) jhatk命令
Jvm Heap Analysis Tool， 与jmap一起使用
示例：
jhat dump.tmp Reading from dump.tmp... Dump file created Tue Jun 28 13:55:09 CST 2016
堆文件太大了
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at com.sun.tools.hat.internal.model.Snapshot.makeId(Snapshot.java:586)
at com.sun.tools.hat.internal.model.Snapshot.addHeapObject(Snapshot.java:166)
at com.sun.tools.hat.internal.parser.HprofReader.readInstance(HprofReader.java:737)
at com.sun.tools.hat.internal.parser.HprofReader.readHeapDump(HprofReader.java:484)
at com.sun.tools.hat.internal.parser.HprofReader.read(HprofReader.java:238)
at com.sun.tools.hat.internal.parser.Reader.readFile(Reader.java:92)
at com.sun.tools.hat.Main.main(Main.java:159)
- 遇到的问题
Attaching to process ID 3538, please wait...
Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process
解决方法:
1. echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
该方法在下次重启前有效。
2. 永久有效方法
sudo vi /etc/sysctl.d/10-ptrace.conf
编辑下面这行:
kernel.yama.ptrace_scope = 1
修改为:
kernel.yama.ptrace_scope = 0
重启系统，使修改生效。
该bug详细信息请参考官方文档: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7050524
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at com.sun.tools.hat.internal.model.Snapshot.makeId(Snapshot.java:586)
at com.sun.tools.hat.internal.model.Snapshot.addHeapObject(Snapshot.java:166)
at com.sun.tools.hat.internal.parser.HprofReader.readInstance(HprofReader.java:737)
at com.sun.tools.hat.internal.parser.HprofReader.readHeapDump(HprofReader.java:484)
at com.sun.tools.hat.internal.parser.HprofReader.read(HprofReader.java:238)
at com.sun.tools.hat.internal.parser.Reader.readFile(Reader.java:92)
at com.sun.tools.hat.Main.main(Main.java:159)