[jvm series-11] jvm performance tuning --- basic use of command line tools

JVM series overall column

---

content	link address
[1] Getting to know virtual machines and java virtual machines	https://blog.csdn.net/zhenghuishengq/article/details/129544460
[2] The class loading subsystem of jvm and the basic use of jclasslib	https://blog.csdn.net/zhenghuishengq/article/details/129610963
[3] The virtual machine stack, program counter, and local method stack of the private area at runtime	https://blog.csdn.net/zhenghuishengq/article/details/129684076
[4] Heap and escape analysis of the shared area of ​​the data area at runtime	https://blog.csdn.net/zhenghuishengq/article/details/129796509
[5] The method area and constant pool of the runtime data area shared area	https://blog.csdn.net/zhenghuishengq/article/details/129958466
[6] Object instantiation, memory layout and access positioning	https://blog.csdn.net/zhenghuishengq/article/details/130057210
[7] Execution engine, interpreter, JIT instant compiler	https://blog.csdn.net/zhenghuishengq/article/details/130088553
[8] Proficient in the underlying mechanism of String	https://blog.csdn.net/zhenghuishengq/article/details/130154453
[9] The underlying principles and algorithms of garbage collection and the basic use of JProfiler	https://blog.csdn.net/zhenghuishengq/article/details/130261481
[10] Types of garbage collectors and internal execution principles	https://blog.csdn.net/zhenghuishengq/article/details/130261481
[11] Basic use of command line tools for jvm performance tuning	https://blog.csdn.net/zhenghuishengq/article/details/130641456

---

1. JVM monitoring and tuning tools

Starting from this article, officially enter the stage of jvm performance tuning

1. Overview of jvm tuning

1.1, possible problems in the production environment

• What should I do if a memory leak occurs in the production environment?
• How much memory should be allocated to the server in the production environment?
• How to tune the performance of the garbage collector?
• How to solve the high CPU load in the production environment?
• How much memory should the production environment allocate for references?
• Without adding logs, how to determine whether a certain line of code is executed?
• How to check the input parameters and return value of a method in real time without adding logs?

Therefore, it is necessary to solve these problems through tuning: prevent OOM, solve OOM, and reduce the frequency of Full GC

1.2, steps for performance optimization

1. Performance monitoring (finding problems): use some monitoring tools to find problems, and then optimize the content of the generated problems. Generally, the problems that may occur include: frequent GC, high CPU load, OOM, memory leaks, deadlocks, The program takes a long time to respond, etc.

2. Performance analysis (troubleshooting): You can analyze the heap logs through GCviewer; you can also use some command line tools, such as jstack, jmap, jinfo, etc.; you can also dump the heap files, and then use the memory analysis tool to analyze the files; Or directly AkiraArthas , or jconsole , JViaual to view real-time JVM status; or directly use jstack to view stack information

3. Performance tuning (problem solving): Appropriately increase memory; optimize code; increase machines; reasonably set the number of thread pool threads; increase message middleware, cache, message queue, etc.

1.3, performance test indicators

Mainly analyze from the following parameters

1. Pause time: the time between submitting a request and returning a request response, generally more concerned about the average time

2. Throughput: the amount of work completed per unit time

3. Concurrency: the number of requests that actually interact with the server at the same time

4. Memory occupation: the memory size occupied by the Java heap area

In tuning, pause times and throughput are the main concerns.

2. JVM monitoring tool

2.1，JPS

Java Process Status, indicating the status of the java process. Through this command, you can display all the HotSpot virtual machine processes in the specified system, which can be used to query the running virtual machine threads.

[root@VM-12-3-centos study]# jps
27586 Jps
29658 jar

In addition to using this jps to view, you can also add the following parameters after jps, namely -l, -q, -m, -v, these parameters are rarely used in daily development, of course, these parameters can be used in combination besides being used alone.

//输出应用程序主类的全名称
[root@VM-12-3-centos study]# jps -l
29658 study-0.0.1-SNAPSHOT.jar
30190 sun.tools.jps.Jps
//只能够看到进程id
[root@VM-12-3-centos study]# jps -q
29658
30190
//查看进程id以及mian方法中参数信息
[root@VM-12-3-centos study]# jps -m
29658 jar
3437 Jps -m
//列出虚拟机启动时的JVM参数
[root@VM-12-3-centos study]# jps -v
29658 study-0.0.1-SNAPSHOT.jar
30190 sun.tools.jps.Jps

You can also use the following command to view the details of the corresponding process number

ps -ef | grep 29658
ps -aux | grep 29658

If -XX:-UsePerfDatathis , then jps will not be able to view the information of this process, and it also shows that this parameter is enabled by default.

2.2, jstat (emphasis)

Jvm Statistics Monitoring Tool , which indicates information about monitoring the running status of the virtual machine. It can display running data such as class loading, memory, garbage collection, and JIT compilation in local or remote virtual machine processes, and is often used to detect garbage collection problems and memory leaks.

After entering the jstat command, you can see how to use it and the parameters that can be added.

[root@VM-12-3-centos study]# jstat
invalid argument count
Usage: jstat -help|-options
       jstat -<option> [-t] [-h<lines>] <vmid> [<interval> [<count>]]
       //<option> : 只要监视的对象信息，如class
       //-t：程序执行的总时间，显示在head头部的Timestamp中
       //-h：多少周期打印一次表头信息
       //<interval> : 周期，比如多长时间打印一次
       //<count> : 打印的总次数，默认只查询一次

This option can be an object of many choices. Take classfor example , it is mainly used to display related information about class loading. This means that the class object is being monitored. Adding -t will display how long a running time has been through this Timestamp in front. -h3 means that the header information will be printed every 3 times, followed by the process number. The next one is 1000 5, which means printing every 1 second, and printing 5 times in total.

[root@VM-12-3-centos study]# jstat -class -t -h3 29658 1000 5
Timestamp     Loaded    Bytes    Unloaded    Bytes      Time   
6913.2         10912   19626.3       0        0.0       6.44
6914.3         10912   19626.3       0        0.0       6.44
6915.3         10912   19626.3       0        0.0       6.44
Timestamp     Loaded    Bytes    Unloaded    Bytes      Time   
6916.3  	   10912   19626.3       0        0.0       6.44
6917.3  	   10912   19626.3       0        0.0       6.44

Next, an explanation of the above parameters

//Timestamp:程序运行的总时间
//Loaded：加载的类的个数  //Bytes：加载类的字节数
//Unloaded：卸载的类的个数 //Bytes：卸载的类的字节数
//Time：加载的总体时间

The above option has only used class, in addition to class, you can also use parameters related to garbage collection

• -gc: Display GC-related heap information. Such as the capacity of the new generation and the old generation, the used space, etc.
• -gccapacity: The displayed content is basically the same as that of -gc, and the content of concern is the maximum and minimum space used by each area of ​​the Java heap
• -gcutil: The display content is basically the same as -gc, and the content concerned is the percentage of the used space to the total space
• -gccause: Same as the -gcutil function, it will additionally output the cause of the last or current gc
• -gcnew: Display the GC status of the new generation
• -gcnewcapacity: basically the same as -gcnew, the output mainly focuses on the maximum and minimum space used
• -geold: display the GC status of the old age
• -gcoldcapacity: The displayed content is basically the same as -gcold, and the output focus is the drunk and minimum space used
• -gcpermcapacity: Display the maximum space used by the permanent generation

Next, you can use this command to analyze the specific parameters related to GC

jstat -gc 进程号id
[root@VM-12-3-centos ~]# jstat -gc 29658
 S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT   
1024.0 1024.0  0.0   480.3  15872.0   5225.1   80384.0    71941.6   59992.0 56814.9 7808.0 7262.8   3727   11.990   3      0.321   12.312

Next, a detailed explanation of the above parameters

//上面的S表示survivor区，C表示已经使用的容量，U表示已经使用的容量
//E表示Eden区，O表示old区，M表示方法区，CCS表示压缩
//YGC表示Young Gc次数，YGCT表示YoungGC时间
//FGC表示Full Gc次数，FGCT表示Full GC时间
//GCT表示总GC的时间，这里指的是Young GC和Full Gc的和

These can still be used with parameters

jstat -gc -t -h3 29658 1000 5

The result is as follows, which is the same as the phenomenon of the above class

[root@VM-12-3-centos ~]# jstat -gc -t -h3 29658 1000 5
Timestamp        S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT   
        86023.0 1024.0 1024.0  0.0   368.3  15872.0   6335.5   80384.0    71965.6   59992.0 56814.9 7808.0 7262.8   3753   12.074   3      0.321   12.395
        86024.0 1024.0 1024.0  0.0   368.3  15872.0   6335.5   80384.0    71965.6   59992.0 56814.9 7808.0 7262.8   3753   12.074   3      0.321   12.395
        86025.0 1024.0 1024.0  0.0   368.3  15872.0   6337.5   80384.0    71965.6   59992.0 56814.9 7808.0 7262.8   3753   12.074   3      0.321   12.395
Timestamp        S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT   
        86026.0 1024.0 1024.0  0.0   368.3  15872.0   6337.5   80384.0    71965.6   59992.0 56814.9 7808.0 7262.8   3753   12.074   3      0.321   12.395
        86027.0 1024.0 1024.0  0.0   368.3  15872.0   6339.6   80384.0    71965.6   59992.0 56814.9 7808.0 7262.8   3753   12.074   3      0.321   12.395

Therefore, in actual development, the memory can be monitored through this jstat . And you can compare the startup time (TimeStamp) of this Java process with the GC time (GCT) to get the ratio of GC time to the total running time. If the ratio exceeds 20%, it means that the pressure on the heap is high. If the ratio exceeds 90% %, it means that there is almost no free space in the heap, and OOM may occur at any time.

2.3，Jinfo

Configuration Info for Java, view information about virtual machine configuration parameters, and can also be used to adjust virtual machine configuration parameters . In many cases, Java applications do not specify all the parameters of the Java virtual machine, and developers may not know the default value of a specific parameter, so developers can easily find the parameters of the Java virtual machine directly through the jinfo tool the current value of .

The basic usage of jinfo is as follows:

[root@VM-12-3-centos study]# jinfo --help
Usage:
   jinfo [option] <pid>
       (to connect to running process)
   jinfo [option] <executable <core>
       (to connect to a core file)
   jinfo [option] [server_id@]<remote server IP or hostname>
       (to connect to remote debug server)

where <option> is one of:
   -flag <name>         to print the value of the named VM flag
   -flag [+|-]<name>    to enable or disable the named VM flag
   -flag <name>=<value> to set the named VM flag to the given value
   -flags               to print VM flags
   -sysprops            to print Java system properties
   <no option>          to print both of the above
   -h | -help           to print this help message

In jinfo, it can mainly be used to view and modify configuration parameters. The specific commands for viewing can be as follows

jinfo -flags PID  //查看一些曾经赋值过的一些参数
jinfo -flags 具体参数 PID  //查看某个java进程的一些参数的值
jinfo -flags UseParallel 29658  //查看当前进程是否用的是Parallel垃圾回收器

You can also modify the parameters. The specific commands for modification are as follows

jinfo -flag 具体参数 PID
jinfo -flag PrintGcDetails 29658  //修改这个打印GC日志的状态，默认不打印，修改后为打印

2.4，jmap

JVM Memory Map, export memory image file, display memory usage. Get the dump file, and you can also get the memory-related information of the Java process, such as the usage of various areas of the heap, statistical information of heap objects, class loading information, etc. You can view related instructions through jmap -help

[root@VM-12-3-centos ~]# jmap -help
Usage:
    jmap [option] <pid>(to connect to running process)
    jmap [option] <executable <core>(to connect to a core file)
    jmap [option] [server_id@]<remote server IP or hostname>(to connect to remote debug server)

jmap can -dumpgenerate a Java heap dump snapshot dump file, -dump: live only saves the surviving objects in the heap. You can use this file to check what causes the heap overflow, memory leak and other issues. Can be generated manually, as follows

jmap -dump:format=b,file=zhs.hprof  29658
jmap -dump:live:format=b,file=../zhs.hprof  29658    //只保存那一刻的存活对象

It can also be generated automatically, and a dump file will be automatically generated when the system is about to OOM. The instructions are as follows

-Xmx100m -XX:HeapDumpOnOutOfMemoryError -XX:heapDumpPath=zhs.hprof

You can -heapoutput detailed information of the entire heap space, including GC usage, heap configuration information, and memory usage information, etc., which is very useful

jmap -heap 29658

You can -histooutput the statistical information in the heap object, including some classes, the number of instances and the total capacity, etc.

jmap -histo 29658
jmap -histo:live 29658   //只统计堆中存活的对象

You can -finalizerinfodisplay the object information that needs to be recycled, but this command is only valid under Linux

jmap -finalizerinfo 29658  

Summary: Since jmap will access the objects in the heap, in order to ensure that it will not be disturbed during this process, jmap needs to access the data at the time of the safe point mechanism, that is, STW, which may lead to deviations in the analysis results of the snapshot. And if it has been unable to stw, then jmap will wait forever. Compared with jstat, jstat is monitored in real time, and its accuracy in this respect is higher than that of jmap.

2.5、jhat

JVM Heap Analysts Tool , obviously, from these few words, it is a tool for analyzing heap memory. In the jmap tool, a dump file can be generated, and this jhat is used in conjunction with this jmap. In jhat, a small server is embedded in it. After generating the analysis result of the dump file, the user can view the analysis result in the browser.

Using this jhat command is equivalent to starting an http service, and the access address is: localhost:7000. However, it has been deleted after JDK9 and 10. It is officially recommended to use the VisualVM graphical interface instead of this command line tool.

2.6, jstack (emphasis)

jmap is used to print a snapshot of the heap space, and this jstack is used to print a snapshot in this stack frame. JVM StackTrace , used to generate a thread snapshot of the specified process at the current moment. The thread snapshot refers to the collection of method stacks being executed by each thread in the virtual machine.

The main functions of generating thread snapshots are as follows: It can be used to locate the reasons for long pauses in threads, such as inter-thread deadlocks, infinite loops, requests for external resources that cause long-term waiting, blocking and other issues.

Its basic syntax is as follows:

[root@VM-12-3-centos study]# jstack 
Usage:
    jstack [-l] <pid>(to connect to running process)
    jstack -F [-m] [-l] <pid>(to connect to a hung process)
    jstack [-m] [-l] <executable> <core>(to connect to a core file)
    jstack [-m] [-l] [server_id@]<remote server IP or hostname>(to connect to a remote debug server)

Options:
    -F to force a thread dump. Use when jstack <pid> does not respond (process is hung)
    -m  to print both java and native frames (mixed mode)
    -l  long listing. Prints additional information about locks
    -h or -help to print this help message

Next, continue to experience it, directly through this jstack -pid

[root@VM-12-3-centos study]# jstack 29658
"C1 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007f0790141000 nid=0x73e3 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007f079013e000 nid=0x73e2 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
...

As follows to simulate a deadlock problem, the code is as follows

package com.zhs.study.test;

/**
 * @author zhenghuisheng
 * @date : 2023/5/12
 */
public class DeadBlockTest {
    
    
    public static void main(String[] args) {
    
    
        StringBuilder s1 = new StringBuilder();
        StringBuilder s2= new StringBuilder();
        new Thread(){
    
    
            @Override
            public void run() {
    
    
                synchronized (s1){
    
    
                    s1.append("a");
                    s1.append("b");
                    try {
    
    
                        Thread.sleep(10000);
                    } catch (InterruptedException e) {
    
    
                        e.printStackTrace();
                    }
                    synchronized (s2){
    
    
                        s2.append("c");
                        s2.append("d");
                    }
                }
            }
        }.start();

        new Thread(){
    
    
            @Override
            public void run() {
    
    
                synchronized (s2){
    
    
                    s1.append("a");
                    s1.append("b");
                    try {
    
    
                        Thread.sleep(10000);
                    } catch (InterruptedException e) {
    
    
                        e.printStackTrace();
                    }
                    synchronized (s1){
    
    
                        s2.append("c");
                        s2.append("d");
                    }
                }
            }
        }.start();
    }
}

After running the above piece of code, get the process number corresponding to the jvm through jps, and then pass jstack + process number. For example, the process number I found by entering jps is 12308, and then jstack 12308I can And other issues.

Found one Java-level deadlock:
=============================
"Thread-1":
  waiting to lock monitor 0x000000001ce7f608 (object 0x000000076b5b6a28, a java.lang.StringBuilder),
  which is held by "Thread-0"
"Thread-0":
  waiting to lock monitor 0x000000001ce7cd78 (object 0x000000076b5b6a70, a java.lang.StringBuilder),
  which is held by "Thread-1"

Java stack information for the threads listed above:
===================================================
"Thread-1":
        at com.zhs.study.test.DeadBlockTest$2.run(DeadBlockTest.java:42)
        - waiting to lock <0x000000076b5b6a28> (a java.lang.StringBuilder)
        - locked <0x000000076b5b6a70> (a java.lang.StringBuilder)
"Thread-0":
        at com.zhs.study.test.DeadBlockTest$1.run(DeadBlockTest.java:23)
        - waiting to lock <0x000000076b5b6a70> (a java.lang.StringBuilder)
        - locked <0x000000076b5b6a28> (a java.lang.StringBuilder)

Found 1 deadlock.

In addition to deadlocks, other sleeps waiting for a long time, etc., can also be viewed.

2.7，jcmd

After jdk7, a new command-line tool jcmd is added, which can implement all commands except jstat, such as exporting heap, memory usage, viewing java process, exporting thread information, executing GC, jvm running time, etc. And it is more recommended to use jcmd instead of jmap.

You can jcmd -lget all the jvm processes and jcmd pid helpview all the commands supported by the specified thread, and you can use this jcmd pid 具体命令to display the data of the instructions and commands of the specified process.

You can first use this help command to see which commands can be executed

jcmd 1744 help

Its execution results are as follows

C:\Users\p'v>jcmd 1744 help
1744:
The following commands are available:
JFR.stop
JFR.start
JFR.dump
JFR.check
VM.native_memory
VM.check_commercial_features
VM.unlock_commercial_features
ManagementAgent.stop
ManagementAgent.start_local
ManagementAgent.start
GC.rotate_log
Thread.print
GC.class_stats
GC.class_histogram
GC.heap_dump
GC.run_finalization
GC.run
VM.uptime
VM.flags
VM.system_properties
VM.command_line
VM.version
help

Then view the data of the corresponding command through the detected command

// 打印线程信息
jcmd 1744 Thread.print
// 查看gc的内存信息
jcmd 1744 GC.class_histogram
// 生成GC的dump文件
jcmd 1744 GC.heap_dump d:\\a.hprof
...