Chapter 22 Overview of Performance Monitoring and Tuning
JVM explained by Song Hongkang from Shang Silicon Valley: bilibili link
1 Dachang interview questions
-
Alipay
- Alipay three sides: what has been done in JVM performance tuning?
-
Millet
- Have you done JVM memory optimization?
- Talk about optimization ideas from four aspects: SQL, JVM, architecture, and database
-
Ant Financial
- JVM compilation optimization
- What has been done in JVM performance tuning
- Which JVM diagnostic tools have been used?
- Two sides: how to tune the JVM, how much proper heap memory and stack space are set
- Three sides: Which JVM-related analysis tools have been used? What are the specific performance optimization steps?
-
Byte beating
- Three sides: how to tune the JVM and how to adjust the parameters?
-
Pinduoduo
- Talk about optimization ideas from four aspects: SQL, JVM, architecture, and database
-
Jingdong
- Which JVM diagnostic tools have been used?
- Why do GCs happen frequently in the spike system with hundreds of thousands of concurrent transactions per second?
- How to optimize the JVM for the daily average million-level trading system?
- How to monitor, locate and solve the online production system OOM?
- How to optimize the performance of a highly concurrent system based on the G1 garbage collector?
2 Background description
2.1 Problems in the production environment
- How to deal with memory overflow in the production environment?
- How much memory should be allocated to the server in the production environment?
- How to tune the performance of the garbage collector?
- How to deal with the high CPU load in the production environment?
- How many threads should the production environment allocate to the application?
- Without logging, how to determine whether a certain line of code has been executed in the request?
- Without log, how can I view the return value of a certain method in real time?
2.2 Why should we optimize?
- Prevent OOM
- Solve OOM
- Reduce the frequency of Full GC
2.3 Considerations at different stages
- Before going online
- Project operation phase
- OOM appears online
3 Overview of tuning
3.1 Basis for monitoring
- Run log
- Exception stack
- GC log
- Thread snapshot
- Heap dump snapshot
3.2 The general direction of tuning
- Write reasonable code
- Fully and reasonable use of hardware resources
- Reasonable JVM tuning
4 Steps to optimize performance
-
The first step (discovering problems): performance monitoring
An activity that collects or views application performance data in a non-forced or intrusive manner . Monitoring usually refers to a preventive or proactive activity implemented in a production, quality assessment, or development environment . When application stakeholders raise performance issues but do not provide enough clues , first we need to perform performance monitoring, followed by performance analysis.
- Frequent GC
- CPU load is too high
- UNCLE
- Memory leak
- Deadlock
- Program response time is longer
-
The second step (troubleshooting): performance analysis
An intrusive way to collect the response results of operating performance problems, it will affect the throughput or responsiveness of the application. Performance analysis is the answer to performance questions, and the focus is usually more focused than performance monitoring. Performance analysis is rarely performed in a production environment. It is usually performed in a quality assessment, system test, or development environment. It is a step after performance monitoring.
- Print the GC log and analyze the log information through GCviewer or gceasy
- Flexible use of command line tools, jstack, jmap, jinfo, etc.
- Dump out the heap file, use the memory analysis tool to analyze the file
- Use Ali Arthas or jconsole, JVisualVM to view JVM status in real time
- jstack view stack information
-
The third step (solving the problem): performance tuning
An activity that changes parameters, source code, and attribute configuration to improve application responsiveness or throughput. Performance tuning is an activity after performance monitoring and performance analysis.
The purpose of performance tuning : reduce the frequency of GC, obtain greater throughput and lower latency with less memory
- Appropriately increase memory and choose garbage collector according to business background
- Optimize code and control memory usage
- Increase the machine, disperse the node pressure
- Reasonably set the number of threads in the thread pool
- Use middleware to improve program efficiency, such as caching, message queues, etc.
- other…
5 Performance indicators/test indicators
-
Pause time (or response time)
The time used between submitting a request and returning a response to the request is generally concerned with response time.
-
List of response times for common operations
operating Response time Open a site A few seconds Query a record in the database (with index) Ten milliseconds Mechanical hard disk addressing once 4 milliseconds Read 1M data sequentially from the mechanical hard disk 2 milliseconds Read 1M data sequentially from SSD disk 0.3 milliseconds Read a piece of data from remote distributed to Redis 0.5 milliseconds Read 1M data from memory Ten microseconds Java program local method call A few microseconds Network transmission 2Kb data 1 microsecond - In the garbage collection link, pause time: the time during which the worker thread of the program is paused when the garbage collection is performed. -XX:MaxGCPauseMills
-
-
Throughput
- A measure of the amount of work (requests) completed per unit of time
- In GC: the ratio of the time to run user code to the total running time (total running time: program running time + memory recovery time) throughput is 1-1/(1+n). -XX:GCTimeRatio=n
-
Concurrency
At the same time, the number of requests that actually interacted with the server.
1000 people are online at the same time, it is estimated that the number of concurrency is between 5%-15%, that is, the amount of concurrent concurrent: between 50-150.
-
Memory footprint
The amount of memory occupied by the Java heap area
-
Mutual relationship
Take highway traffic conditions as an example.
- Throughput: the number of toll stations passing through the expressway every day (can also be understood as the high-speed tolls charged by the toll stations)
- Concurrent number: the number of vehicles on the highway
- Corresponding time: vehicle speed
Few cars, fast speeds, low charges ----> Few concurrency, fast response time, low throughput
Appropriately increase the number of vehicles, the speed is faster, and the charges are more ----> the number of concurrent connections is appropriate, the response time is faster, and the throughput is larger
Too many cars, slow speeds, low charges ----> Too many concurrency, slow response time, low throughput