In a production environment, we can not pass breakpoint debugging, the new log, visualization tools to immediately view the current operating status and get an error message, this time, with the Java comes with command-line tools, and related dump analysis tools as well as some small tips that can greatly enhance the efficiency of our investigation of the problem
Operating parameters
The following lists some of the common and very effective command and parameters to view information about a Java program is running, so as to assist you understand the procedure running. There are a large number of functions provided by other available parameters, see themselves oracle documentation
View JVM arguments
jps -l View all Java programs running simultaneously display start class class name, get to the PID
1 4706 org.apache.catalina.startup.Bootstrap 2 5023 sun.tools.jps.Jps
jinfo -flags PID View runtime JVM parameters and process parameters
1 Attaching to process ID 28987, please wait... 2 Debugger attached successfully. 3 Server compiler detected. 4 JVM version is 25.171-b11 5 Non-default VM flags: -XX:CICompilerCount=3 -XX:InitialHeapSize=132120576 -XX:MaxHeapSize=2092957696 -XX:MaxNewSize=697303040 -XX:MinHeapDeltaBytes=524288 -XX:NewSize=44040192 -XX:OldSize=88080384 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseParallelGC 6 Command line: -Dspring.config.location=application.properties -Dspring.profiles.active=staging
java -XX:+PrintFlagsFinal -version View the current virtual machine default JVM parameters
View real-time status GC
jstat -gc PID 1000 10 View a gc information per second, a total of 10 times
Explained more output parameters, each field reference https://docs.oracle.com/javase/8/docs/technotes/tools/unix/jstat.html
1 S0C S1C S0U S1U EC Inc OC MU or MC CCC CCSU YGC GCT YGCT FGC GCTF 2 512.0 512.0 15.3 0.0 4416.0 1055.2 11372.0 7572.5 14720.0 14322.5 1664.0 1522.8 40 0.137 8 0.039 0.176
May encounter during the prompt sun.jvm.hotspot.runtime.VMVersionMismatchException: Supported versions are 24.181-b01. Target VM is 25.171-b11problem because multiple versions are installed, use which, ls -lcan be positioned to the same Profile currently executing Java programs Java version
Error investigation
Memory Problems
Memory leaks cause OOM? Unusually high memory usage? This is a production environment is so often the problem, Java provides a dump file for us of what happened in a recorded memory, we need to use some tools from which to obtain valuable information.
Export Dump File
- In advance of the Java program plus these parameters printed dump file
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./ - For running programs using jmap :
jmap -dump:format=b,file=heap.hprof PID
Dump file analysis
If the Dump file is not too large, it can spread to http://heaphero.io/index.jsp to analyze
Files are large, and you want to carry out a more systematic analysis, recommended MAT analysis, there are several common View
- Home] in [Leak Suspects can figure out where the problem lies
- Click [Create a histogram from an arbitrary set of objects found in the number of all objects]
- Right-click on an object [Merge Shortest Paths to GC Roots] -> [exclude all phantom / weak / soft etc. references] can query an object to a large number of references which GC ROOT
Threading Issues
Task for a long time not to withdraw? CPU load is too high? Probably because of an infinite loop or deadlock, leading some threads have been executed without interruption, but no error is the most annoying, so logs can not see the error message, but nor dump file and analysis, because nothing to do with memory. This time we need to use thread analysis tools to help us.
Export jstack file
Use jstack PID > 文件if it fails please add -Fparameters, if it fails, use the user to use a Java program starts execution jstack, the following is part of the output format jstack
1 线程名 PID的16进制 2 "http-nio-8080-Acceptor-0" #17 daemon prio=5 os_prio=0 tid=0x00007fac2c4bd000 nid=0x29f4 runnable [0x00007fac192f6000] 3 java.lang.Thread.State: RUNNABLE（tomcat的工作线程正在运行，有NEW/RUNNABLE/BLOCKED/WAITING/TIMED_WATING/TERMINATED状态） 4 at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) 5 at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) 6 at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) 7 - locked <0x00000000faf845a8> (a java.lang.Object) 8 at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:682) 9 at java.lang.Thread.run(Thread.java:748)
output jstack can see all the threads and their state, we can see which threads are running our own have created, it is likely that the threads have been executed, this time thread name it particularly important, it is recommended that you specify a meaningful thread name when creating a new thread. Of course, look through the PID is also very convenient.
Troubleshooting steps
topView java program to which high loadtop -p PID -HView the status of all processes running process- Under high load the recording thread ID,
printf "&x" PIDconverted to hexadecimal jstack PID > 文件- Converted into a thread ID query thread running stack after the hex file with the jstack
- From where the thread stack learned in the implementation of what tasks, combined with the code of business judgment