Performance Test and the waste disposer JVM tuning popularity

Basics of GC

1. What is garbage

C language application memory: malloc free

C++： new delete

c / C ++ manually reclaims memory

Java: new?

Automatic memory reclamation, simple programming, the system is not error-prone, manual release of memory, two types of easy questions:

1. Forget recovery
2. Recycled many times

No references to an object or plurality of objects (circular references)

2. How to locate trash

1. Reference count (the ReferenceCount)
2. Root up algorithm (RootSearching)

3. Common garbage collection algorithm

1. Clear mark (mark sweep) - discontinuous position fragmentation efficiency is low (two passes)
2. Copy algorithm (copying) - free of debris, waste of space
3. Compression mark (mark compact) - no fragmentation, low efficiency (twice scanning, necessary to adjust the pointer)

4.JVM generational memory model (for generational garbage collection algorithm)

1. Model portions of the garbage collector uses

GC addition Epsilon ZGC Shenandoah are logical model generational

G1 is logic generational, regardless of the physical substituting

Generational addition only logical, and physical generational

1. + + New generation of permanent year old-generation (1.7) Perm Generation / metadata area (1.8) Metaspace

   1. Metadata Generation permanent - Class
   2. Generation permanent must specify the size limit, the metadata may be provided, may not be provided, without limit (limited by the physical memory)
   3. String constants 1.7 - permanent generation 1.8 - heap
   4. MethodArea logical concept - Generation of permanent, metadata

2. New Generation = Eden + 2 th region suvivor

   1. After YGC recovery, most of the objects will be recovered alive into the s0
   2. YGC again, alive object eden + s0 -> s1
   3. YGC again, eden + s1 -> s0
   4. Old enough -> years old (15 CMS 6)
   5. s area fit -> years old

3. Years old

   1. Diehards
   2. Years old and full of FGC Full GC

4. GC Tuning (Generation)

   1. Minimize FGC
   2. MinorGC = CVL
   3. MajorGC = FGC

5. Object allocation process of FIG. 

6. Age is not important dynamic :() https://www.jianshu.com/p/989d3b06a49d

7. :( allocation guarantee is not important) survivor space area during YGC is not enough space guarantee direct access to the old year Reference: https: //cloud.tencent.com/developer/article/1082730

5. Common garbage collector

1. JDK born Serial follow to improve efficiency, the birth of the PS, in order to meet CMS, born PN, CMS version 1.4 was introduced late, CMS is a landmark of the GC, it opens the concurrent recovery process, but the problems are more CMS, so now any CMS is a default JDK version concurrent garbage collection because they can not stand STW
2. Serial younger generation serial recovery
3. PS younger generation of parallel recovery
4. ParNew the young generation with concurrent recovery of CMS
5. SerialOld
6. ParallelOld
7. ConcurrentMarkSweep years old concurrent, garbage collection and applications running at the same time, reducing the STW time (200ms) CMS more problems, so now is not a default version of CMS, CMS can only manually specify since it is MarkSweep, will certainly have fragmented the problem, debris reaches a certain level, when the old CMS's assignment object allocation fewer than using SerialOld years old were recovered imagine: PS + PO -> add memory to change the garbage collector -> PN + CMS + SerialOld (a few hours - STW days) memory, threaded recovered dozens of G -> G1 + FGC tens G -> T memory on the server ZGC algorithm: three color marker + Incremental Update
8. G1 (10ms) Algorithm: Three Color Marker + SATB
9. ZGC (1ms) PK C++ 算法：ColoredPointers + LoadBarrier
10. Shenandoah 算法：ColoredPointers + WriteBarrier
11. Eplison
12. PS and PN differentiated Further reading: ▪ https://docs.oracle.com/en/java/javase/13/gctuning/ergonomics.html#GUID-3D0BB91E-9BFF-4EBB-B523-14493A860E73
13. Relations with the garbage collector memory size
    1. Serial tens of megabytes
    2. PS hundreds of megabytes - several G
    3. CMS - 20G
    4. G1 - hundreds of G
    5. ZGC - 4T - 16T (JDK13)

1.8 The default garbage collection: PS + ParallelOld

Common garbage collector combination of parameters setting: (1.8)

• -XX:+UseSerialGC = Serial New (DefNew) + Serial Old
  • Small programs. By default, this option is not, HotSpot is automatically selected according to the calculated collector and configuration and JDK version
• -XX:+UseParNewGC = ParNew + SerialOld
  • This combination is rarely used (in some versions obsolete)
  • https://stackoverflow.com/questions/34962257/why-remove-support-for-parnewserialold-anddefnewcms-in-the-future
• -XX:+UseConc(urrent)MarkSweepGC = ParNew + CMS + Serial Old
• -XX:+UseParallelGC = Parallel Scavenge + Parallel Old (1.8默认) 【PS + SerialOld】
• -XX:+UseParallelOldGC = Parallel Scavenge + Parallel Old
• -XX:+UseG1GC = G1

• Linux did not find in the default view of GC method, and windows will print UseParallelGC

  • java +XX:+PrintCommandLineFlags -version
  • To distinguish by GC log

• What Linux version 1.8 default garbage collector in the end is?

  • 1.8.0_181 default (can not see) Copy MarkCompact
  • 1.8.0_222 default PS + PO

JVM tuning the first step to understand the common JVM command line parameters

• VM command line arguments Reference: https: //docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html

• HotSpot classification parameters

Standard: - at the beginning, all of the HotSpot support

Non-standard: the beginning of the -X, a specific version of HotSpot supports a particular command

Unstable: beginning -XX, the next version may be canceled

1. Distinguish concepts: memory leak memory leak, memory overflow out of memory
2. java -XX:+PrintCommandLineFlags HelloGC
3. java -Xmn10M -Xms40M -Xmx60M -XX:+PrintCommandLineFlags -XX:+PrintGC HelloGC PrintGCDetails PrintGCTimeStamps PrintGCCauses
4. java -XX:+UseConcMarkSweepGC -XX:+PrintCommandLineFlags HelloGC
5. java -XX: + PrintFlagsInitial default parameter values
6. java -XX: + PrintFlagsFinal final parameter value
7. java -XX: + PrintFlagsFinal | grep xxx find the corresponding parameters
8. java -XX:+PrintFlagsFinal -version |grep 

Before tuning basic concepts:

1. Throughput: Time user code / (User code execution time garbage collection time +)
2. Response time: STW The shorter, the better the response time

The so-called tuning, first determine the pursuit of what? Priority throughput, or response time priority? Also in the case of satisfying a certain response time, much required to achieve a certain ...

problem:

Scientific computing throughput. Data mining, thrput. General throughput priority: (PS + PO)

Response time: Web GUI API (1.8 G1)

What is tuning?

1. JVM planning and conduct pre-tuning on demand
2. Optimal operation JVM runtime environment (slow, Caton)
3. JVM process to solve problems occurred in operation (OOM)

Tuning, from planning to start

• Tuning, starting from the business scene, tuning, no business scene of bullying

• No monitoring (stress tests, the results can be seen), not tuning

• step:

  1. Familiar with the business scene (not the best garbage collector, only the most suitable garbage collector)
     1. Response time, pause time [CMS G1 ZGC] (as needed in response to a user)
     2. Throughput = the user time / (user time time + GC) [the PS]
  2. Select combination recovery
  3. Computing memory requirements (experience 1.5G 16G)
  4. Selected CPU (the higher the better)
  5. Set's size, age, upgrades
  6. Set the log parameters
     1. -Xloggc:/opt/xxx/logs/xxx-xxx-gc-%t.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=20M -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCCause
     2. Or generate a log file every day
  7. Observe the log case

Improve the environment

1. There is a 500,000 PV information websites (extract documents from disk into memory) the origin server 32, 1.5G heap, user feedback, the site is relatively slow, so the company decided to upgrade, the new server is 64, 16G heap memory, Caton result of user feedback is very serious, but less efficiently than before the
   1. Why the original site slow? Many users browse data, a lot of data load into memory, out of memory, frequent GC, STW long, slow response time
   2. Why more Caton? The larger the memory, the longer the time FGC
   3. Supposed to? PS -> PN + CMS, or G1
2. The system CPU always 100%, how to tune? (Interview HF) CPU100% then there must be a thread in the system resources,
   1. Find out which process cpu high (top)
   2. Which thread the high cpu process (top -Hp)
   3. Export of the thread's stack (jstack)
   4. Find out which method (stack frame) elapsed time (jstack)
   5. High proportion of worker threads | high proportion of the garbage collection thread
System memory soar, how to find the problem?
1. Export heap memory (jmap / jvisualvm tool)
2. We use the article about the JVM monitoring tools into the file for analysis
3. How to monitor JVM
   1. We use the article about the JVM monitoring tools to monitor

Solve problems JVM running

1. Operation and maintenance team is usually the first to suffer alarm information (CPU Memory)

2. top command problem observed: Memory growing high CPU utilization

3. top -Hp observe the threads in the process, which thread high proportion of CPU and memory

4. jps java process jstack locate specific positioning thread status, focusing on:. WAITING BLOCKED eg waiting on < 0x0000000088ca3310> (a java.lang.Object) If there is a process in 100 threads, many threads are ON Waiting  , be sure to find which thread holds the lock how to find? Jstack dump of information search, find  , which thread holds the lock RUNNABLE see

5. Orbit gc situation jvisualvm, see YGC and FGC speed

6. jmap - histo port | head -20, look how many objects produced

7. Online system, memory is particularly large, have a significant impact on the process during execution jmap, even Caton (electric providers are not suitable) 1: set the parameters HeapDump, OOM time will automatically generate heap dump file 2: Many servers backup (high availability), turning off this server does not affect other servers  3: online positioning (usually something small company with less than)

8. Jvisualvm were analyzed using a dump file 

9. Find the problem code

GC algorithm based on the concept of

• Card Table Since when do YGC, OLD need to scan the entire area, the efficiency is very low, so the JVM designed CardTable, if there is a OLD area CardTable target point Y area, it will be set Dirty, next scan, just scan Dirty Card in the structure, Card Table achieved by BitMap

CMS

CMS questions

1. Memory Fragmentation

-XX: + UseCMSCompactAtFullCollection -XX: CMSFullGCsBeforeCompaction defaults to 0 refers to the number of times through the FGC before compression

1. Floating Garbage

Concurrent Mode Failure 产生：if the concurrent collector is unable to finish reclaiming the unreachable objects before the tenured generation fills up, or if an allocation cannot be satisfiedwith the available free space blocks in the tenured generation, then theapplication is paused and the collection is completed with all the applicationthreads stopped

Solution: lower the threshold to trigger the CMS

PromotionFailed

Similar solution for keeping old's enough space

-XX: CMSInitiatingOccupancyFraction 92% can reduce this value, so that CMS's old enough space

G1

1. ▪https://www.oracle.com/technical-resources/articles/java/g1gc.html

Case Summary

OOM produced a variety of reasons, some programs may not produce OOM, constantly FGC (CPU soared high, but the memory recovery extremely rare) (top case)

1. Hardware upgrade the system but the problem Caton

2. Improper use of the thread pool generated OOM issues continue to List of Riga objects (too LOW)

3. smile jira issue the actual system restart to solve the problem continues to add memory to replace the garbage collector + G1 really where the problem? do not know

4. tomcat http-header-size too large problem (Hector)

5. lambda expressions method results in overflow problem

1. Rewrite finalize caused frequent GC millet cloud, HBase synchronization system, alarm system timeout nginx access, the last investigation, C ++ programmers to rewrite finalize caused frequent question why the GC C ++ programmers will rewrite finalize? (New delete) finalize takes a long (200ms)

2. If there is a system, memory consumption has been less than 10%, but observe GC logs, FGC always found frequently produce, what would cause? System.gc () (this comparison Low)

3. Distuptor have a chain length can be set, if too large, then a large object, the consumer does not take the initiative to complete the release, will overflow (from the dead style)

4. With jvm will overflow, mycat with collapse before, a temporary version 1.6.5 parsing sql subqueries algorithm, the United sql 9 Ge exists would result in the generation of millions of objects (from the dead style)

5. new number of threads, will produce native thread OOM, (low) should use the thread pool solution: reduce heap space (too TMlow a) set aside more memory to produce native thread JVM memory accounted for 50% the proportion of physical memory - 80%

GC common parameters

• -Xmn -Xms -Xmx -Xss the young generation minimum heap maximum stack space
• -XX: + UseTLAB use TLAB, by default
• -XX: + PrintTLAB printing the usage of TLAB
• -XX: TLABSize set the size TLAB
• -XX: + DisableExplictGC System.gc () does not work, FGC
• -XX:+PrintGC
• -XX:+PrintGCDetails
• -XX:+PrintHeapAtGC
• -XX:+PrintGCTimeStamps
• -XX: + PrintGCApplicationConcurrentTime (low) print application time
• -XX: + PrintGCApplicationStoppedTime (low) when printing long pause
• -XX: + PrintReferenceGC (low importance) recorded a recovery of how many different types of references cited
• -verbose: class class loading process in detail
• -XX:+PrintVMOptions
• -XX: + PrintFlagsFinal -XX: + PrintFlagsInitial must be used
• -Xloggc:opt/log/gc.log
• -XX: MaxTenuringThreshold liter on behalf of age, maximum 15
• Spin lock times -XX: PreBlockSpin hot code detecting parameters -XX: CompileThreshold escape analysis scalar replacement is not recommended these settings ...

Parallel common parameters

• -XX:SurvivorRatio
• -XX: PreTenureSizeThreshold large object in the end how much
• -XX:MaxTenuringThreshold
• -XX: + the number of threads ParallelGCThreads parallel collector, the same applies to CMS, the same general set and the number of CPU cores
• -XX: + UseAdaptiveSizePolicy zones is automatically selected size ratio

CMS common parameters

• -XX:+UseConcMarkSweepGC
• -XX: ParallelCMSThreads number of threads CMS
• -XX: After CMSInitiatingOccupancyFraction what percentage of CMS began collecting old's use, the default is 68% (approximate value), if SerialOld Caton frequent, should adjust small (frequently recovered CMS)
• -XX: + UseCMSCompactAtFullCollection compress when FGC
• How many times after compression FGC CMSFullGCsBeforeCompaction: -XX
• -XX:+CMSClassUnloadingEnabled
• Perm CMSInitiatingPermOccupancyFraction be recovered to what proportion of the time: -XX
• GCTimeRatio set GC run time program or the percentage of time
• -XX: MaxGCPauseMillis pause time, is a suggested time, GC will try to use all means to achieve this time, such as to reduce the young generation

G1 common parameters

• -XX:+UseG1GC
• -XX: MaxGCPauseMillis recommended value, G1 will try to adjust the number of blocks Young district to achieve this value
• -XX: GCPauseIntervalMillis? GC interval
• -XX: + G1HeapRegionSize partition size, the recommended value is gradually increased, 12481632. With increasing size, garbage longer survival, longer intervals GC, GC but each time will be longer ZGC has been improved (Dynamic block size)
• G1NewSizePercent minimum scale new generation, the default is 5%
• G1MaxNewSizePercent the largest proportion of the new generation, the default is 60%
• GCTimeRatio GC time recommended ratio, G1 will be adjusted based on the value of the heap space
• ConcGCThreads number of threads
• InitiatingHeapOccupancyPercent start G1 heap space occupancy ratio

Reference material

1. https://blogs.oracle.com/jonthecollector/our-collectors
2. https://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html
3. http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp
4. JVM tuning reference documentation: https: //docs.oracle.com/en/java/javase/13/gctuning/introduction-garbage-collection-tuning.html#GUID-8A443184-7E07-4B71-9777-4F12947C8184
5. jmap Command Reference: https://www.jianshu.com/p/507f7e0cc3a3
   1. jmap -heap pid
   2. jmap -histo pid
   3. jmap -clstats pid