Part 1 of JVM_12_Execution Engine_Silicon Valley

1 Overview of Execution Engine

position

overview

• The execution engine is one of the core components of the Java virtual machine.
• "Virtual machine" is a concept relative to "physical machine". Both machines have code execution capabilities. The difference is that the execution engine of a physical machine is built on the processor, cache, instruction set, and operating system levels. The execution engine of the virtual machine is implemented by software itself , so it can set the instruction set and the structure of the execution engine without being restricted by physical conditions, and can execute which instruction set formats are not directly supported by the hardware.

The relationship between JVM and execution engine

​ The main task of the JVM is to load bytecode into its interior , but bytecode cannot run directly on the operating system, because bytecode execution is not equivalent to local machine instructions, and it contains only some capable Bytecode instructions, symbol tables, and other auxiliary information recognized by the JVM.

​ So, if you want to run a Java program, the task of the execution engine (Execution Engine) is to interpret/compile bytecode instructions into local machine instructions on the corresponding platform . In simple terms, the execution engine of the JVM acts as a translator that translates high-level languages ​​into machine language.

The working process of the execution engine

(1) What kind of bytecode instructions the execution engine needs to execute during execution depends entirely on the PC register.

(2) Whenever an instruction operation is executed, the PC register will update the address of the next instruction that needs to be executed by itself.

(3) Of course, during the execution of the method, the execution engine may accurately locate the instance information stored in the Java heap through the object reference stored in the local variable table, and locate the instance information through the metadata pointer in the object header. Type information of the target object.

• From the appearance point of view, the input and output of the execution engine of all Java virtual machines are consistent: the input is the bytecode binary stream, the processing process is the equivalent process of bytecode parsing and execution, and the output is the execution result.

2 Java code compilation and execution process

Before most of the program code is converted into the target code of the physical machine or the instruction set that the virtual machine can execute, it needs to go through the steps in the above figure.

Java code compilation is done by the Java source code compiler, and the flow chart is as follows:

Question: What is an interpreter (Interpreter), what is a JIT compiler?

Interpreter : When the Java virtual machine starts, it will interpret the bytecode line by line according to the predefined specifications , and when the content in each bytecode file is "translated" into the local machine instructions of the corresponding platform, it will be executed.

JIT : (Just In Time Compole) compiler: It is the virtual machine that directly compiles the source code into the machine language related to the local machine platform.

Question: Why is Java said to be a semi-compiled and semi-interpreted language?

In the era of JDK1.0, it is more accurate to position the Java language as "interpretation and execution". Later, Java also developed a compiler that can directly generate native code.

Now when the JVM executes Java code, it usually combines interpretation and execution with compilation and execution.

Example:

3 Machine code, instructions, assembly language

machine code

• Various instructions expressed in binary encoding are called machine instruction codes . At the beginning, people used it to write programs, which is machine language.
• Although machine language can be understood and received by computers, it is too different from human language to be easily understood and memorized by people, and it is easy to make mistakes when programming with it.
• Once the program written in it is input into the computer, the CPU directly reads and runs it, so it has the fastest execution speed compared with programs written in other languages.
• Machine instructions are closely related to the CPU, so different types of CPUs correspond to different machine instructions.

instruction

• Since the machine code is a binary sequence composed of 0 and 1, the readable lines are too poor, so people invented instructions.
• The instruction is to simplify the specific 0 and 1 sequence in the machine code into the corresponding instruction (generally abbreviated in English, such as mov, inc, etc.), and the readability is slightly better.
• Because different hardware platforms perform the same operation, the corresponding machine codes may be different, so the corresponding machine codes for the same instruction (such as mov) on different hardware platforms may also be different.

Instruction Set

• Different hardware platforms have different supported instructions. Therefore, the instructions supported by each platform are called the instruction set of the corresponding platform.
• as usual
  • The x89 instruction set corresponds to the x86 architecture platform
  • ARM instruction set, corresponding to the platform of ARM architecture

Assembly language

• Because the readability of the instructions is still too poor, people invented assembly language.
• In assembly language, mnemonics (Mnemonics) are used to replace the opcodes of machine instructions , and address symbols (Symbol) or labels (Label) are used to replace the addresses of instructions or operands .
• On different hardware platforms, assembly language corresponds to different machine language instruction sets, which are converted into machine instructions through the assembly process.
  • Since computers only understand instruction codes, programs written in assembly must also be translated into machine instruction codes before computers can recognize and execute them.

high level language

• In order to make programming easier for computer users, various high-level computer languages ​​appeared later. The reason why high-level language is closer to human than machine language and assembly language
• When a computer executes a program written in a high-level language, it still needs to interpret the program and program the machine's instruction code . The program that completes this process is called an interpreter or a compiler.

bytecode

• Bytecode is a binary code (file) in an intermediate state (intermediate code). It is more abstract than machine code. It needs to be translated before it can be called a miracle.
• Bytecode is mainly to achieve specific software operation and software environment, and has nothing to do with the hardware environment .
• The implementation of bytecode is through compilers and virtual machines. The compiler compiles the source code into bytecode, and the virtual machine on a specific platform translates the bytecode into instructions that can be directly executed.
  • A typical application of own code is Java bytecode.

C, C++ source program execution process:

The compilation process can be divided into two phases: compilation and assembly.

• Compilation process: read the source program (character stream), analyze it lexically and grammatically, and convert high-level language instructions into functionally equivalent assembly codes
• Assembly process: actually refers to the process of translating assembly language code into target machine instructions.

4 interpreter

overview

​ The original intention of JVM designers is simply to meet the cross-platform characteristics of Java programs , so they avoid using static compilation to directly generate local machine instructions, thus giving birth to the realization that the compiler uses line-by-line compilation bytecode execution at runtime. program idea.

Interpreter working mechanism (or work task)

• The role of the interpreter in the true sense is a runtime "translator", which "translates" the content in the bytecode file into the local instruction execution of the corresponding platform.
• After a bytecode instruction is interpreted and executed, the operation is then interpreted according to the next bytecode instruction that needs to be executed recorded in the PC register.

Interpreter classification

​ In the development history of Java, there are two sets of interpretation executors, namely the ancient bytecode interpreter and the now commonly used template interpreter.

• The bytecode interpreter simulates the execution of the bytecode through pure software code during execution , which is very inefficient.
• The template interpreter associates each bytecode with a template function , and the template function directly generates this machine code during execution, which greatly improves the performance of the interpreter.
  • In HotSpot VM, the interpreter is mainly composed of Interpreter module and Code module.
    • Interpreter module: implements the core functions of the interpreter
    • Codo module: used to manage the native machine instructions generated by the HotSpot VM at runtime

status quo

• Because the interpreter is very simple in design and implementation, in addition to the Java language, there are many high-level languages ​​that are also executed based on the interpreter, such as Python, Perl, Ruby, etc. But today, interpreter-based execution has become synonymous with inefficiency and is often ridiculed by C/C++ programmers.
• In order to solve this problem, the JVM platform supports a technology called just-in-time compilation. The purpose of just-in-time compilation is to prevent the function from being interpreted and executed, but to compile the entire function into machine code. Every time the function is executed, only the compiled machine code is executed. That is , this method can greatly improve the execution efficiency.
• But in any case, the execution mode based on the interpreter still made an indelible contribution to the development of the intermediate language.

5 JIT Compiler

Execution Classification of Java Code

• The first is to compile the source code into a bytecode file, and then convert the bytecode file into machine code execution through the interpreter at runtime
• The second is to compile and execute (directly compiled into machine code). In order to improve execution efficiency, modern virtual machines use just-in-time compilation technology (JIT, Just In Time) to compile methods into machine codes before executing them.

status quo

• HotSpot VM is one of the representative works of high-performance virtual machines currently on the market. It adopts an architecture in which an interpreter and a just-in-time compiler coexist . When the Java virtual machine is running, the interpreter and the just-in-time compiler can write each other, learn from each other, and try to choose the most appropriate way to save the time of compiling native code and the time of directly interpreting and executing code.
• Today, the operating performance of Java programs has been reborn, and has reached the point where it can compete with C/C++ programs.

question:

​ Some developers will feel surprised, since HotSpot VM already has a built-in JIT compiler, why do you need to use an interpreter to "drag" the execution performance of the program? For example, JRockit VM does not contain an interpreter inside, and all bytecodes are compiled and executed by a just-in-time compiler.

​ First clear:

​ When the program starts, the interpreter can take effect immediately, saving the compilation time and executing it immediately.

​ In order for the compiler to function, it takes a certain amount of execution time to compile the code into local code. However, after compiling to native code, the execution efficiency is high.

So:

​ Although the execution performance of the JRockit VM program will be very high, the program will inevitably take longer to compile when it starts. For server-side applications, startup time is not the focus, but for some application scenarios that are concerned about startup time, it may be necessary to adopt an architecture where an interpreter and a just-in-time compiler coexist in exchange for a balance point.

​ In this mode, when the Java virtual machine starts, the interpreter can play a role first, and wait for the instant compiler to compile before executing, which can save a lot of unnecessary compilation time. Over time, the compiler comes into play, compiling more and more code into native code to achieve higher execution efficiency.

​ At the same time, interpreted execution is used as an "escape door" for the compiler when the aggressive optimization of the compiler is not established

Execution method of HotSpot JVM

​ When the virtual machine is started, the interpreter can play its role first , without having to wait for the instant compiler to compile and then execute, which can save a lot of unnecessary compilation time . And as the running time of the program goes on, the just-in-time compiler gradually plays a role, and compiles valuable bytecodes into local machine instructions according to the hotspot detection function in exchange for higher program execution efficiency.

Specific case

​ Pay attention to the subtle dialectical relationship between interpreted execution and compiled execution in the online environment. The load that the machine can withstand in the hot state is greater than that in the cold state . If the traffic in the hot state is used to cut the flow, the server in the cold state may be suspended because the server cannot carry the traffic.

​ In the release process of the production environment, the release is performed in batches, divided into multiple batches according to the number of machines, and the number of machines in each batch accounts for at most 1/8 of the entire cluster. There was once such a failure case: a programmer released in batches on the publishing platform, and when entering the total number of releases, he mistakenly filled in the division into two batches for release. If it is in a hot engine state, under normal circumstances, the general machine can barely carry the traffic, but because the newly started JVM is interpreted and executed, and hot code statistics and JIT dynamic compilation have not yet been performed, after the machine is started, the current 1/2 release is successful All of the servers immediately crashed, this failure shows the existence of JIT, —— Ali team

JIT compiler

Concept explanation:

• The "compiler" of the Java language is actually an "uncertain" operation process, so it may refer to a front-end compiler (actually called "compiler front-end" is more accurate) that converts .java files into .class files. process
• May also refer to the virtual machine's backend runtime compiler (JIT compiler)
• It may also refer to the process of directly compiling .class files into local machine code using a static ahead-of-time compiler (AOT compiler, Ahead Of Time Compoler).

Front-end compilers: Sun's Javac, the incremental compiler (ECJ) in Eclipse JDT.

JIT Compiler: C1 C2 Compiler for HotSpot VM.

AOT compilers: GNU Compiler for the Java (GCJ), Excelsior JET.

how to choose?

Hot code and detection method

​ Of course, whether it is necessary to start the JIT compiler to directly compile the bytecode into the local machine instructions of the corresponding platform depends on the frequency of the code being called and executed . Regarding those bytecodes that need to be compiled into local, also known as "hot codes" , the JIT compiler will make deep optimizations for those frequently called "hot codes" at runtime , and directly compile them into corresponding Platform machine instructions to improve the execution performance of Java programs.

• A method that is called many times, even a loop body with a large number of loops inside the method body can be called "hot code" , because it can be compiled into local machine instructions by the JIT compiler. Because of the execution process of this compilation method, it is also called on-stack replacement, or OSR (On Stack Replacement) compilation for short.
• How many times does a method have to be called , or how many times does a loop body need to be executed to meet this standard? A clear threshold is necessary for the JIT compiler to compile these "hot codes" into local machine instructions for execution. This mainly relies on the hotspot detection function .
• Currently, the hotspot detection method used by the HotSpot VM lock is counter-based hotspot detection.
• Using counter-based hotspot detection, HotSpot VM will create two different types of counters for each method, namely method call counter (Invocation Counter) and assembly counter (Back Edge Counter).
  • The method call counter is used to call the number of times the method was called.
  • The edge counter is used to count the number of loops executed by the loop body

method call counter

• This counter is used to count the number of times the method is called. Its default threshold is 1500 times in Client mode and 10000 times in Server mode. Exceeding this threshold triggers JIT compilation.
• This threshold can be artificially set through the virtual machine parameter -XX:CompileThreshold .
• When a method is called, it will first check whether there is a JIT-compiled version of the method, and if it exists, it will be executed with limited use of compiled native code. If there is no compiled version, add 1 to the value of the call counter of this method, and then judge whether the sum of the value of the method call counter and the return edge counter exceeds the threshold of the method call counter. If the threshold has been exceeded, a code compilation request for that method is submitted to the just-in-time compiler.

heat decay

• If no settings are made, the method call counter counts not the absolute number of times the method is called, but a relative execution frequency, that is, the number of times the method is called within a period of time . When a certain time limit is exceeded , if the number of method calls is still not enough to submit it to the just-in-time compiler for compilation, the call counter of this method will be reduced. Generally , this process is called method call counter heat decay (Counter Decay) , and this period of time is called the half-life cycle (Count Half Lift Time) of submethod statistics .
• The action of heat decay is performed by the way when the virtual machine is garbage collected. You can use the virtual machine parameter -XX:-UseCounterDecay to turn off the heat decay and let the method counter count the absolute number of method calls. In this way, as long as the system runs long enough , most of the methods will be compiled into native code.
• In addition, you can use the -XX:CounterHalfLifeTime parameter to set the time of the half-life cycle in seconds

Back edge counter

Its function is to count the number of times the loop body code is executed in a method, and the instruction that jumps after encountering the control flow in the bytecode is called "Back Edge". Apparently, the purpose of resume counting is to trigger OSR compilation.

HotSpot VM can set the program execution mode

​ By default, the HotSpot VM adopts an architecture in which an interpreter and a compiler coexist. Of course, developers can explicitly specify for the Java virtual machine through commands according to specific application scenarios whether to use the interpreter to execute completely or completely Execute with a just-in-time compiler . As follows:

• -Xint : Execute the program completely in interpreter mode;
• -Xcomp : Execute the program completely in just-in-time compiler mode. If there is a problem with just-in-time compilation, the interpreter will step in to execute.
• -Xmixed : Use the mixed mode of interpreter + just-in-time compiler to execute the program together. (default)

specific practice

# 查看使用的是哪种方式
>java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
# 修改执行方式
java -Xint -version
# 再次查看
java -version

Specific time-consuming code demo:

/**
 * -Xint : 花费的时间为：7636
 * -Xcomp : 花费的时间为：1020
 * -Xmixed 花费的时间为：1090
 *
 */
public class IntCompTest {
  public static void main(String[] args) {
    long start = System.currentTimeMillis();
    testPrimeNumber(1000_000);

    long end = System.currentTimeMillis();

    System.out.println("花费的时间为：" + (end - start));
  }

  public static void testPrimeNumber(int count) {
    for (int i = 0; i < count; i++) {
      // 计算100以内的质数
      label:
      for (int j = 2; j <= 100; j++) {
        for (int k = 2; k < Math.sqrt(j); k++) {
          if (j % k == 0) {
            continue label;
          }
        }
      }
    }
  }
}

JIT classification in HotSpot VM

​ There are two JIT compilers embedded in the HotSpot VM, namely Client Compiler and Server Compiler, but in most cases we simply call them C1 compiler and C2 compiler. Developers can use the following command to display which kind of just-in-time compiler is used by the specified Java virtual machine at runtime, as follows:

• -client : Specifies that the Java virtual machine runs in Client mode and uses the C1 compiler;
  • The C1 compiler performs simple and reliable optimization of the bytecode in a fraction of the time . Faster compilation speed has been achieved.
• -server : Specify Java to run in Server mode and use the C2 compiler.
  • The C2 compiler takes a long time to optimize, as well as aggressive optimization . But optimized code executes more efficiently.

64-bit operating system can only use Server mode

Different optimization strategies of C1 and C2 compilers

• Different compilers have different optimization strategies. The C1 compiler mainly includes method inlining, devirtualization, and redundancy elimination.
  • Method inlining: Compile the referenced function code to the reference point, which can reduce the generation of stack frames, parameter passing and jumping process.
  • Devirtualization: inlining the only implementation class
  • Redundancy Elimination: Fold code that will not be executed during runtime
• The optimization of C2 is mainly at the global level, and escape analysis is the basis of optimization. Based on escape analysis, there are the following optimizations in C2:
  • Scalar Swap: Replaces property values ​​of aggregated objects with scalar values
  • On-stack allocation: for unescaped objects allocations are made to the object on the stack rather than the heap
  • Synchronization Elimination: Clear Synchronization Operation: Usually the value is synchronized

layered compilation

​Tiered Compilation strategy : Program interpretation and execution (without performance monitoring) can trigger C1 compilation, compiling bytecode into machine code, willing to perform simple optimization, and performance monitoring can also be added. C2 compilation will be based on performance Monitor information for radical optimization.

However, after the Java7 version, once the developer displays the specified command "-server" in the program, the hierarchical compilation strategy will be enabled by default, and the C1 compiler and the C2 compiler will write together to execute the compilation.

Summarize

• Generally speaking, the performance of the machine compiled by JIT is higher than that of the interpreter.
• The startup time of the C2 compiler is slower than that of the C1 compiler. After the system runs stably, the execution speed of the C2 compiler is much faster than that of the C1 compiler.

Write at the end 1:

• Since JDK10, HotSpot has added a new just-in-time compiler: the Graal compiler.

• The compilation effect has been reviewed by the C2 compiler in just a few years. Future can be expected

• Currently, with the "Experimental Status" tab, you need to use the switch parameter

  -XX:+UnlockExperimentalVMopions -XX:+UseJVMCICompiler deactivated before it can be used

Written at the end 2: About the AOT compiler

• jdk9 introduces the AOT compiler (static ahead of time compiler, Ahead Of Time Compiler)
• Java 9 introduces the experimental AOT compilation tool jaotc. It uses the Graal compiler to convert the input Java class file into machine code and store it in the generated dynamic shared library.
• The so-called AOT compilation is a concept opposite to just-in-time compilation. We know that just-in-time compilation refers to the process of converting bytecode into machine code that can be run directly on the hardware during , and deploying it to the hosting environment. AOT compilation refers to the process of converting bytecode into machine code before the program runs .
• The biggest advantage: Java virtual machine loading has been precompiled into a binary library, which can be executed directly. There is no need to wait for the warm-up of the just-in-time compiler, reducing the bad experience of "running slowly for the first time" brought by Java applications.
• shortcoming:
  • It breaks Java's "compile once, run everywhere", and must compile the corresponding release package for each different hardware and OS
  • The dynamics of the Java linking process is reduced , and the loaded code must be fully known by the compiler.
  • Need to continue to optimize, initially support Linux x64 java base