How to solve the CPU or memory surge in production

Locating CPU Elevation

Method 1:
1-Start: java -jar 2_cpu-0.0.1-SNAPSHOT.jar 8 > log.file 2>&1 &
2-Generally speaking, the application server usually only deploys java applications, you can check with top first, whether It is caused by the java application:
Command: top 3- If so, check the java entry ID,
Command: jps -l 4- Find out the best non-CPU thread in the process,
Command: top -Hp pid 25128 5- Change the thread Convert the ID to hexadecimal,
command: printf “%x\n” Thread ID 623c 25148 6-Export java stack information, search the result according to the thread ID in the previous step: Command: jstack 11976 >stack.txt grep 2ed7 stack.txt - A 20 Method
2: Online tool: https://gceasy.io/ft-index.jsp 1-The snapshot file exported in Method 1, just upload it to this website

memory problem

Memory leak
refers to the heap memory that has been dynamically allocated in the program is not released or cannot be released for some reason, resulting in a waste of system memory, resulting in
serious consequences such as slowing down the running speed of the program or even system crashes.
General memory leaks:

1. Frequent memory leaks: The code with memory leaks will be executed multiple times, and each time it is executed, it will cause a memory leak.
2. Occasional memory leaks: codes with memory leaks only occur in certain specific environments or operating procedures. Frequent and sporadic are relative. For a specific environment, sporadic may become regular. So the test environment and test method are crucial to detect memory leaks.
3. One-time memory leak: The code with a memory leak will only be executed once, or due to an algorithmic defect, there will always be one and only one memory leak.
4. Implicit memory leak: The program keeps allocating memory during the running process, but does not release the memory until the end. Strictly speaking, there is no memory leak here, because eventually the program releases all the allocated memory. But for a server program that needs to run for days, weeks or even months, not releasing the memory in time may eventually exhaust all the memory of the system. Therefore, we call this type of memory leak an implicit memory leak. From the point of view of the user using the program, the memory leak itself will not cause any harm. As an ordinary user, it does not feel the existence of the memory leak at all. The real danger is the accumulation of memory leaks, which can eventually eat up all the memory in the system. From this point of view, a one-time memory leak is not harmful because it does not accumulate, while an implicit memory leak is very harmful because it is more difficult to detect than frequent and sporadic memory leaks .

Memory leaks in JAVA:

The above description is the usual way of memory leaks, and of course it also applies to java, but for java, the problem seems to become simpler. An important feature of JAVA is to automatically manage the recovery of memory through the garbage collector (GC), and There is no need for the programmer to free the memory himself. In theory, all memory occupied by objects that will no longer be used in Java can be reclaimed by GC, but Java also has memory leaks, but its performance is different from C or C++. This is usually caused by poor design, and therefore can be avoided by design. The fundamental question is whether the object we need to control is not destroyed when it should be destroyed, or the increment of the object is not expected to exceed our expectations.
1-Object growth exceeds expectations
2-Design objects that should be destroyed, and resident memory
For problem one, give a common design rule: the creation of the thread pool should show that the specified blocking queue is small, to avoid the default value out of control, which is extremely bad In this case, a large number of threads are created, resulting in OOM.
The second problem often occurs in the design cache, stored map, list, infinite growth, out of control.

Common points that easily lead to memory leaks

1-Thread pool creation does not display the specified blocking queue size
2-Forget to recycle objects in the management of ThreadLocal
3-All places involving resource links, do not forget to close resources
4-Class member variables are collections, or there are in singleton mode Collections refer to a large number of other objects
5-java method, whether to pass by value or pass by reference, resulting in a small period of time, the memory is not recycled as expected

Memory overflow (Out Of Memory)

Memory overflow is memory out of bounds. A very common situation of memory out of bounds is call stack overflow (that is, stackoverflow), although this situation can be regarded as a manifestation of insufficient stack memory. The difference between memory overflow and memory leak: Memory overflow: When applying for memory, the JVM does not
have
enough memory space.
Memory leak: memory is applied for but not released, resulting in waste of memory space

JVM parameters

1-JVM parameter type

1.1 Standard parameters: -version, -help
1.2 X parameters (knowledge): -Xint, -Xmixed
1.3 XX parameters:
1.3.1 boolean type: -XX:+PrintGCDetails
1.3.2 KV setting value type: -XX:MetaspaceSize= 128m
2- View memory parameters
2.1 -XX:+PrintFlagsInitial Mainly view the initial default (not dependent on java process) case:java -XX:+PrintFlagsInitial
2.2 -XX:+PrintFlagsFinal Mainly view modification and update (not dependent on java process) case:java - XX:+PrintFlagsFinal 2.3 -XX:+PrintCommandLineFlags Print command line parameters
2.4 jinfo View process-related data case: jinfo -flag MetaspaceSize pid Problem: -Xms: initial size memory, default physical memory 1/64, equivalent to -XX:InitialHeapSize -Xmx: Maximum allocated memory, the default is 1/4 of physical memory, equivalent to -XX:MaxHeapSize -Xss: Set the size of a single thread stack, generally the default is 512k~1024k, equivalent to -XX:ThreadStackSize Reference: https: //docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html

Common configuration

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=heapdump.txt
-XX:+PrintGCDetails
-Xloggc:gc.log

tool

1-jvm gc log analysis tool: https://javagc.cn/ https://gceasy.io/ft-index.jsp
2-Memory snapshot analysis tool: mat, jprofile, VisualVM
3-java comes with:
-jps process View
-jstat: A command-line tool for monitoring various running status information of virtual machines. It can display the class record, memory, garbage collection, JIT compilation and other operating data in the local or remote virtual machine process
jstat -gc pid #garbage collection statistics
jstat -gccapacity pid #heap memory statistics
jstat -gcnew pid #new generation garbage collection
statistics -gcnewcapacity pid #New generation memory statistics
jstat -gcold pid #Old generation garbage collection statistics
jstat -gcoldcapacity pid #Old generation memory statistics j
stat -gcutil pid #Summary garbage collection statistics
jstat -printcompilation pid #JVM compilation method statistics
jstat -class pid #Class loading statistics
-jinfo parameter configuration view
-jmap memory monitoring
jmap -clstats pid #Print the class loader of the process and the persistent generation object information loaded by the class loader
jmap -heap pid #Check the memory usage of the process heap, including the GC used Algorithms, heap configuration parameters, and heap memory usage across generations.
jmap -histo[:live] pid #View the histogram of the number and size of objects in the heap memory, if you bring live, only count live objects
jmap -dump:format=b,file=dumpFileName pid #jmap process memory usage dump to file