JVM memory and garbage collection series: execution engine

Execution engine

Execution engine overview

The execution engine belongs to the lower layer of the JVM, which includes an interpreter, a just-in-time compiler, and a garbage collector

The execution engine is one of the core components of the Java virtual machine. "Virtual machine" is a concept relative to "physical machine". Both machines have code execution capabilities. The difference is that the execution engine of the physical machine is directly built on the processor, cache, instruction set, and operating system levels. , And the execution engine of the virtual machine is realized by the software itself, so it is possible to customize the structure of the instruction set and execution engine without being restricted by physical conditions, and can execute those instruction set formats that are not directly supported by the hardware.

The main task of JVM is to load bytecode into it, but bytecode cannot run directly on the operating system, because bytecode instructions are not equivalent to local machine instructions. It contains only some Bytecode instructions, symbol tables, and other auxiliary information recognized by the JVM.

Then, if you want a Java program to run, the task of the Execution Engine is to interpret/compile bytecode instructions into native machine instructions on the corresponding platform. Simply put, the execution engine in the JVM acts as a translator that translates high-level language into machine language.

The workflow of the execution engine

• What bytecode instructions the execution engine needs to execute during the execution process completely depends on the PC register.
• Whenever an instruction operation is executed, the PC register will update the address of the next instruction that needs to be executed.
• Of course, during the execution of the method, the execution engine may accurately locate the object instance information stored in the Java heap area through the object reference stored in the local variable table, and locate the target object through the metadata pointer in the object header Type information.

From the appearance point of view, the input and output of the execution engine of all Java virtual machines are the same: the input is a bytecode binary stream, the processing process is the equivalent process of bytecode analysis and execution, and the output is the execution process.

Java code compilation and execution process

Before most of the program code is converted into the target code of the physical machine or the instruction set that the virtual machine can execute, it needs to go through the various steps in the above figure

• The orange part in front is the process of generating bytecode files, which has nothing to do with JVM
• The blue and green behind is the process that JVM needs to consider

Java code compilation is done by the Java source code compiler. The flowchart is as follows:

The execution of Java bytecode is completed by the JVM execution engine, the flowchart is shown below

We use a general picture to talk about the interpreter and the compiler

What is Interpreter

When the Java virtual machine is started, the bytecode will be interpreted line by line according to the predefined specifications, and the content in each bytecode file will be "translated" into the local machine instructions of the corresponding platform for execution.

What is an IT compiler

JIT (Just In Time Compiler) compiler: the virtual machine directly compiles the source code into the machine language related to the local machine platform.

Why Java is a semi-compiled and semi-interpreted language

In the JDK1.e era, it is more accurate to position the Java language as "interpretation and execution". Later, Java also developed a compiler that can directly generate native code. Now when JVM executes Java code, it usually combines interpretation and execution with compilation and execution.

After translating the local code, you can do a cache operation and store it in the method area

Machine code, instructions, assembly language

Machine code

Various instructions expressed in binary encoding are called machine instruction codes. In the beginning, people used it to write programs, which is machine language.

Although machine language can be understood and accepted by computers, it is too different from people's languages, it is not easy to be understood and remembered by people, and programming with it is prone to errors.

Once the program written with it is input to the computer, the CPU directly reads and runs it, so it has the fastest execution speed compared with the programs written in other languages.

Machine instructions are closely related to the CPU, so different types of CPUs correspond to different machine instructions.

instruction

Since machine code is a binary sequence composed of 0 and 1, the readability is really poor, so people invented instructions.

The instruction is to simplify the specific sequence of 0 and 1 in the machine code into the corresponding instruction (usually abbreviated in English, such as mov, inc, etc.), which is slightly more readable

Because different hardware platforms perform the same operation, the corresponding machine code may be different, so the same instruction (such as mov) on different hardware platforms may have different machine codes.

Instruction Set

The instructions supported by different hardware platforms are different. Therefore, the instructions supported by each platform are called the instruction set of the corresponding platform.
As common

• x86 instruction set, corresponding to the x86 architecture platform
• ARM instruction set, corresponding to the platform of ARM architecture

Assembly language

Because the readability of the instructions is still too poor, so people invented assembly language.

In assembly language, mnemonics are used to replace the opcodes of machine instructions, and address symbols (Symbo1) or labels (Labe1) are used to replace the addresses of instructions or operands. On different hardware platforms, assembly language corresponds to different machine language instruction sets, which are converted into machine instructions through the assembly process.

Since computers only recognize instruction codes, programs written in assembly language must also be translated into machine instruction codes before the computer can recognize and execute them.

High-level language

In order to make it easier for computer users to program, various high-level computer languages ​​appeared later.

High-level language is closer to human language than machine language and assembly language. When a computer executes a program written in a high-level language, it still needs to interpret and compile the program into machine instruction codes. The program that completes this process is called an interpreter or compiler.

[External link image transfer failed. The source site may have an anti-leech link mechanism. It is recommended to save the image and upload it directly (img-CUQ0wbMr-1606810235780)(https://tuchuang666.oss-cn-shenzhen.aliyuncs.com/img/image -20200710085323733.png)]

High-level languages ​​are not directly translated into machine instructions, but into assembly language, such as C and C++ as mentioned below

C, C++ source program execution process

The compilation process can be divided into two stages: compilation and assembly.

Compilation process: read the source program (character stream), analyze the lexical and grammar, and convert high-level language instructions into functionally equivalent assembly code

Assembly process: actually refers to the process of translating assembly language code into target machine instructions.

Bytecode

Bytecode is a binary code (file) of an intermediate state (intermediate code). It is more abstract than machine code and needs to be translated by a interpreter before it can become machine code.

Bytecode is mainly for realizing specific software operation and software environment, and has nothing to do with hardware environment.

The implementation of bytecode is through a compiler and a virtual machine. The compiler compiles the source code into bytecode, and the virtual machine on a specific platform translates the bytecode into instructions that can be directly executed.

• The typical application of bytecode is: Java bytecode

Interpreter

The original intention of the JVM designers was simply to meet the cross-platform features of Java programs, so they avoided the use of static compilation to directly generate local machine instructions, which gave birth to the implementation of interpreters that use line-by-line interpretation of bytecode execution programs at runtime. Thoughts.

Why are Java source files not directly translated into JMV, but into bytecode files? Probably because the directly translated code is relatively large

The real role of the interpreter is a runtime "translator", which "translates" the contents of the bytecode file into the local machine instructions of the corresponding platform for execution.

After a bytecode instruction is interpreted and executed, the interpretation operation is performed according to the next bytecode instruction recorded in the PC register that needs to be executed.

Interpreter classification

In the development history of Java, there are two sets of interpretation executors, namely the ancient bytecode interpreter and the template interpreter commonly used now.

The bytecode interpreter simulates the execution of bytecode through pure software code during execution, which is very inefficient.

The template interpreter associates each bytecode with a template function, and the template function directly generates the machine code when the bytecode is executed, thereby greatly improving the performance of the interpreter.

In HotSpot VM, the interpreter is mainly composed of Interpreter module and Code module.

• Interpreter module: implements the core functions of the interpreter
• Code module: used to manage the local machine instructions generated by the HotSpot VM at runtime

status quo

Because the interpreter is very simple in design and implementation, in addition to the Java language, there are many high-level languages ​​that are also executed based on the interpreter, such as Python, Per1, Ruby, etc. But today, interpreter-based execution has become synonymous with inefficiency, and is often ridiculed by some C/C++ programmers.

To solve this problem, the JVM platform supports a technology called just-in-time compilation. The purpose of just-in-time compilation is to prevent the function from being interpreted and executed, but to compile the entire function body into machine code. Each time the function is executed, only the compiled machine code is executed. This method can greatly improve the execution efficiency.

However, the execution mode based on the interpreter still makes an indelible contribution to the development of the intermediate language.

JIT compiler

Java code execution classification

The first is to compile the source code into a bytecode file, and then use the interpreter to convert the bytecode file into machine code execution at runtime

The second is to compile and execute (compile directly into machine code). In order to improve execution efficiency, modern virtual machines will use just-in-time compilation technology (JIT, Just In Time) to compile methods into machine code before execution

HotSpot VM is one of the masterpieces of high-performance virtual machines currently on the market. It uses an architecture in which an interpreter and a just-in-time compiler coexist. When the Java virtual machine is running, the interpreter and the just-in-time compiler can cooperate with each other, each learn from each other's strengths, and try to choose the most appropriate way to weigh the time to compile the native code and the time to directly interpret and execute the code.

Today, the running performance of Java programs has long been reborn, and it has reached a point where it can compete with C/C++ programs.

Here comes the problem

Some developers will feel surprised. Since the JIT compiler is already built into the HotSpot VM, why do you need to use an interpreter to "drag" the execution performance of the program? For example, JRockit VM does not include an interpreter, and all bytecodes are compiled and executed by a just-in-time compiler.

• The JRockit virtual machine cuts off the interpreter, that is, only uses the just-in-time compiler. That's because JRockit is only deployed on the server. Generally, there is time for him to compile the instructions. The response is not demanding. After the compilation of the timely compiler is completed, it will provide better performance.

First of all, make it clear:
when the program is started, the interpreter can take effect immediately, saving the time of compilation and executing it immediately.
For the compiler to function, it takes a certain amount of execution time to compile the code into local code. But after compiling to native code, the execution efficiency is high.

Therefore:
Although the execution performance of the program in JRockit VM will be very efficient, it will inevitably take longer to compile when the program starts. For server-side applications, startup time is not the focus of attention, but for those application scenarios that value startup time, it may be necessary to adopt an architecture where an interpreter and a just-in-time compiler coexist in exchange for a balance.

In this mode, when the Java virtualizer is started, the interpreter can play a role first, instead of waiting for the just-in-time compiler to complete the compilation before executing, which can save a lot of unnecessary compilation time. As time goes by, the compiler comes into play, compiling more and more codes into local codes and gaining higher execution efficiency.

At the same time, interpretation and execution serve as the compiler's "escape door" when the compiler's radical optimization fails.

HotSpot JVM execution method

When the virtual machine is started, the interpreter can work first, instead of waiting for the just-in-time compiler to complete the compilation and then execute it, which can save a lot of unnecessary compilation time. And with the passage of program running time, the just-in-time compiler gradually comes into play, according to the hot spot detection function, the valuable bytecode is compiled into local machine instructions in exchange for higher program execution efficiency.

Case study

Pay attention to the subtle dialectic relationship between interpretation and execution and compilation and execution in the online environment. The load that the machine can bear in the warm state is greater than in the cold state. If the flow is cut with the traffic in the warm state, the server in the cold state may be suspended due to the fact that it cannot carry the traffic.

In the production environment release process, release is carried out in batches, divided into multiple batches according to the number of machines, and the number of machines in each batch accounts for at most 1/8 of the entire cluster. There has been a failure case like this: a programmer released in batches on the release platform, and when inputting the total number of release batches, he mistakenly filled in the ingredients as two batches of release. If it is in the hot state, under normal circumstances, half of the machines can barely carry traffic, but because the JVM just started is interpreted and executed, hot code statistics and JIT dynamic compilation have not been performed, resulting in the current 1/2 release after the machine is started. All of the servers went down immediately. This failure indicates the existence of JIT. —Ali team

Concept explanation

• The "compilation period" of the Java language is actually a period of "uncertain" operation, because it may refer to a front-end compiler (actually called "compiler front-end" is more accurate) converting .java files into .class files Process; it may also refer to the back-end runtime compiler of the virtual machine (JIT compiler, Just In Time Compiler)
• The process of converting bytecode into machine code.
• It may also refer to the process of using a static ahead of time compiler (Ahead of Time Compiler) to directly compile .java files into local machine code.

Front-end compiler: Sun's Javac, incremental compiler (ECJ) in Eclipse JDT.

JIT compiler: C1 and C2 compiler of HotSpot VM.

AOT compiler: GNU Compiler for the Java (GCJ), Excelsior JET.

Hot spot detection technology

A method that is called multiple times, or a loop body with a large number of loops in the method body can be called "hot code", so it can be compiled into local machine instructions by the JIT compiler. Since this compilation method occurs during the execution of the method, it is called on-stack replacement, or OSR (On Stack Replacement) compilation for short.

How many times does a method need to be called, or how many times does a loop body need to execute to reach this standard? A clear threshold must be required before the JIT compiler will compile these "hot code" into local machine instructions for execution. Here mainly depends on the hot spot detection function.

The hot spot detection method currently used by HotSpot VM is counter-based hot spot detection.

Using counter-based hot spot detection, HotSpot V will create two different types of counters for each method, namely the Invocation Counter and the Back Edge Counter.

• Method call counter is used to count the number of method calls
• The edge return counter is used to count the number of loops executed by the loop body

Method call counter

This counter is used to count the number of times the method is called. Its default threshold is 1500 times in Client mode and 10000 times in Server mode. If this threshold is exceeded, JIT compilation will be triggered.

This threshold can be set manually through the virtual machine parameter -XX:CompileThreshold.

When a method is called, it will first check whether the method has a JIT-compiled version. If it exists, the compiled native code will be used first. If there is no compiled version, add 1 to the method call counter value, and then determine whether the sum of the method call counter and the return counter value exceeds the method call counter threshold. If the threshold has been exceeded, a code compilation request for this method will be submitted to the just-in-time compiler.

Hot spot attenuation

If nothing is set, the method call counter does not count the absolute number of method calls, but a relative execution frequency, that is, the number of method calls within a period of time. When a certain time limit is exceeded, if the number of method calls is still not enough for it to be submitted to the just-in-time compiler for compilation, the call counter of this method will be reduced by half. This process is called the counter decay of the method call counter. , And this period of time is called the Counter Half Life Time (Counter Half Life Time)

• Half-life cycle is a concept in chemistry. For example, the age of unearthed cultural relics can be obtained by checking C60.

The action of heat decay is performed by the way when the virtual machine is collecting garbage. You can use the virtual machine parameter
-XX:-UseCounterDecay to turn off heat decay, and let the method counter count the absolute number of method calls, so as long as the system runs for a long time , Most methods will be compiled into local code.

In addition, you can use the -XX:CounterHalfLifeTime parameter to set the half-life cycle time, in seconds.

Back-to-edge counter

Its function is to count the number of executions of the loop body code in a method, and the instruction that the control flow jumps backward in the bytecode is called the "back edge". Obviously, the purpose of establishing back-side counter statistics is to trigger OSR compilation.

HotSpotVM can set the program execution method

By default, HotSpot VM adopts an architecture in which an interpreter and a just-in-time compiler coexist. Of course, developers can explicitly specify for the Java virtual machine through commands according to specific application scenarios whether to use the interpreter at runtime or not. It is executed entirely with a just-in-time compiler. As follows:

• -Xint: execute the program completely in interpreter mode;
• -Xcomp: The program is executed entirely in the mode of just-in-time compiler. If there is a problem with just-in-time compilation, the interpreter will intervene in execution
• -Xmixed: Adopt a mixed mode of interpreter + just-in-time compiler to execute programs together.

JIT classification in HotSpotVM

JIT compilers are also divided into two types, namely C1 and C2. There are two JIT compilers embedded in HotSpot VM, namely Client Compiler and Server Compiler, but in most cases we call them C1 compiler and C2 for short. translater. Developers can explicitly specify which kind of just-in-time compiler the Java virtual machine uses at runtime through the following command, as shown below:

• -client: Specify the Java virtual machine to run in Client mode and use the C1 compiler;

  • The C1 compiler performs simple and reliable optimizations on the bytecode, which takes a short time. In order to achieve faster compilation speed.

• -server: Specify the Java virtual machine to run in server mode and use the C2 compiler.

  • C2 performs time-consuming optimizations and radical optimizations. But optimized code execution is more efficient. (Using C++)

Different optimization strategies for C1 and C2 compilers

There are different optimization strategies on different compilers. On the C1 compiler, there are mainly methods for inlining, de-virtualization, and elimination of surplus.

• Method inlining: Compile the referenced function code to the reference point, which can reduce the generation of stack frames, reduce parameter passing and jump processes
• Devirtualization: inline the only implementation fan
• Redundancy elimination: fold out some code that will not be executed during operation

The optimization of C2 is mainly at the global level, and escape analysis is the basis of optimization. There are several optimizations on C2 based on escape analysis:

• Scalar replacement: replace the attribute value of the aggregate object with a scalar value
• Allocation on the stack: For objects that have not escaped, the objects are allocated on the stack instead of the heap
• Synchronization elimination: clear synchronization operation, usually referred to as synchronized

Hierarchical Compilation Strategy

Tiered Compilation (Tiered Compilation) strategy: Program interpretation and execution (without performance monitoring enabled) can trigger C1 compilation and compile bytecode into machine code. Simple optimization can be carried out, or performance monitoring can be added. C2 compilation will be based on performance monitoring. Information is radically optimized.

However, after the Java7 version, once the developer explicitly specifies the command "-server" in the program, the hierarchical compilation strategy will be turned on by default, and the C1 compiler and C2 compiler will cooperate with each other to perform compilation tasks.

to sum up

• Generally speaking, the performance of machine code compiled by JIT is better than that of an interpreter.
• The startup time of the C2 compiler is slower than that of C1. After the system is stably executed, the execution speed of the C2 compiler is much faster than that of the C1 compiler.

AOT compiler

jdk9 introduced the AoT compiler (static ahead of time compiler, Ahead of Time Compiler)

Java 9 introduced the experimental AOT compilation tool aotc. It uses the Graal compiler to convert the input Java class files into machine code and store them in the generated dynamic shared library.

The so-called AOT compilation is a concept opposite to just-in-time compilation. We know that just-in-time compilation refers to the process of converting bytecode into machine code that can be directly run on the hardware during the running of the program and deploying it to the hosting environment. AOT compilation refers to the process of converting bytecode into machine code before the program runs.

.java -> .class -> (使用jaotc) -> .so

The biggest benefit: Java virtual machine loading has been pre-compiled into a binary library and can be executed directly. There is no need to wait for the compiler to warm up in time, which reduces the bad experience of "first run slow" for Java applications.

Disadvantages:

• Destroy the java "compile once, run everywhere", you must compile the corresponding release package for each different hardware and OS
• The dynamic nature of the Java linking process is reduced, and the loaded code must all be known in the compiler.
• Still need to continue to optimize, initially only supports Linux X64 java base

Write to the end

• Since JDK10, HotSpot has added a new just-in-time compiler: Graal compiler
• Compilation effect The G2 compiler has been reviewed in just a few years, and the future can be expected
• At present, with the experiment status label, you need to use the switch parameter to activate it to use

-XX:+UnlockExperimentalvMOptions -XX:+UseJVMCICompiler