Almost all Java programmers know Spring. Its IoC (Dependency Inversion) and AOP (Aspect-Oriented Programming) functions are very powerful and easy to use. The bytecode generation technology behind it (the technology to modify and generate Java bytecode as needed during runtime) is an important supporting technology.
Java bytecode can be interpreted and executed on the JVM (Java Virtual Machine), or compiled and executed on the fly. In fact, in addition to Java, many languages such as Groovy, Kotlin, Closure, Scala, etc. on the JVM also need to generate bytecode. In addition, playscript can also generate bytecode to run efficiently on the JVM!
Moreover, bytecode generation technology is useful. You can use it to compile high-level languages into bytecode, and you can also inject new code into the original code to implement functions such as performance monitoring.
Currently, I have a need for a practical project. One of our products requires a rules engine to parse custom DSL and calculate rules. The amount of data processed by this rule engine is relatively large, so the higher its performance, the better. Therefore, it would be ideal if the DSL was compiled into bytecode.
Since bytecode generation technology has strong practical value, today, I will take you to master it.
I will first take you to understand Java's virtual machine and bytecode instructions, then use the ASM tool to generate bytecode, and finally, compile from AST into bytecode. Through such a process, you will deepen your understanding of the Java virtual machine, master the bytecode generation technology, thereby better understanding the operating mechanism of Spring, and even have the ability to write such a tool!
Java virtual machine and bytecode
Bytecode is an intermediate code in binary format. It is not the target code of the physical machine, but runs on the Java virtual machine and can be interpreted and executed and compiled and executed on the fly.
When talking about back-end technology, I emphasize how to generate binary code that runs directly on the computer, which is more in line with statically compiled languages such as C, C++, and Go. But if you want to interpret and execute, apart from directly interpreting and executing AST, I have not talked about other interpretation and execution techniques.
At present, the more common language for interpretation and execution is to use a virtual machine, the most typical of which is the JVM, which can interpret and execute Java bytecode.
There are two technologies for the design of virtual machines: one is a stack-based virtual machine; the other is a register-based virtual machine.
The standard JVM is a stack-based virtual machine (hereinafter referred to as "stack machine").
Each thread has a JVM stack, and each time a method is called, a stack frame is generated to support the execution of this method. The stack frame also contains the local variable array (including method parameters and local variables), the operand stack and the constants used by this method. The design of this stack frame is actually very similar to the structure of the stack frame in C language that we have learned before.
The stack machine performs calculations based on the operand stack. Taking the calculation of "2+3" as an example, just convert it into a reverse Polish expression, "2 3 +", and then execute it in sequence. That is: first push 2 onto the stack, then push 3 onto the stack, and then execute the addition instruction. At this time, the two operands are popped from the stack for addition calculation, and then the result is pushed onto the stack.
As you can see, the addition instruction of the stack machine does not need to have operands. It is just a simple "iadd". This is different from the IR you have learned before. why? Because the operands are all on the stack, the addition operation requires 2 operands, and just pop 2 elements from the stack.
In other words, the operands of the instruction are determined by the stack, and we do not need to explicitly specify the storage location for each operand, so the instructions can be shorter, which is an advantage of the stack machine.
Next, let’s talk about the characteristics of bytecode.
What does bytecode look like? I wrote a simple class in which the foo() method implements a simple addition calculation. You can see what its corresponding bytecode looks like:
public class MyClass {
public int foo(int a){
return a + 3;
}
}
Type the following two lines of commands in the command line terminal to generate a bytecode file in text format:
javac MyClass.java
javap -v MyClass > MyClass.bc
Open the MyClass.bc file and you will see the following snippet:
public int foo(int);
Code:
0: iload_1 //把下标为1的本地变量入栈
1: iconst_3 //把常数3入栈
2: iadd //执行加法操作
3: ireturn //返回
Among them, the foo() method has a total of four instructions. The first three instructions are to calculate an addition expression a+3. This is performed exactly in the order of the reverse Polish expression: first push a local variable onto the stack, then push the constant 3 onto the stack, and then perform the addition operation.
如果你细心的话，应该会发现： 把参数a入栈的第一条指令，用的下标是1，而不是0。这是因为，每个方法的第一个参数（下标为0）是当前对象实例的引用（this）。
我提供了字节码中，一些常用的指令，增加你对字节码特点的直观认识，完整的指令集可以参见 JVM的规格书：
其中，每个指令都是8位的，占一个字节，而且iload_0、iconst_0这种指令，甚至把操作数（变量的下标、常数的值）压缩进了操作码里，可以看出，字节码的设计很注重节省空间。
根据这些指令所对应的操作码的数值，MyClass.bc文件中，你所看到的那四行代码，变成二进制格式，就是下面的样子：
你可以用“hexdump MyClass.class”显示字节码文件的内容，从中可以发现这个片段（就是橙色框里的内容）：
现在，你已经初步了解了基于栈的虚拟机， 与此对应的是基于寄存器的虚拟机。 这类虚拟机的运行机制跟机器码的运行机制是差不多的，它的指令要显式地指出操作数的位置（寄存器或内存地址）。 它的优势是： 可以更充分地利用寄存器来保存中间值，从而可以进行更多的优化。
例如，当存在公共子表达式时，这个表达式的计算结果可以保存在某个寄存器中，另一个用到该公共子表达式的指令，就可以直接访问这个寄存器，不用再计算了。在栈机里是做不到这样的优化的，所以基于寄存器的虚拟机，性能可以更高。而它的典型代表，是Google公司为Android开发的Dalvik虚拟机和Lua语言的虚拟机。
这里你需要注意， 栈机并不是不用寄存器，实际上，操作数栈是可以基于寄存器实现的，寄存器放不下的再溢出到内存里。只不过栈机的每条指令，只能操作栈顶部的几个操作数，所以也就没有办法访问其它寄存器，实现更多的优化。
现在，你应该对虚拟机以及字节码有了一定的了解了。那么，如何借助工具生成字节码呢？你可能会问了：为什么不纯手工生成字节码呢？当然可以，只不过借助工具会更快一些。
就像你生成LLVM的IR时，也曾获得了LLVM的API的帮助。所以，接下来我会带你认识ASM这个工具，并借助它为我们生成字节码。
字节码生成工具ASM
其实，有很多工具会帮我们生成字节码，比如Apache BCEL、Javassist等，选择ASM是因为它的性能比较高，并且它还被Spring等著名软件所采用。
ASM 是一个开源的字节码生成工具。Grovvy语言就是用它来生成字节码的，它还能解析Java编译后生成的字节码，从而进行修改。
ASM解析字节码的过程，有点像XML的解析器解析XML的过程：先解析类，再解析类的成员，比如类的成员变量（Field）、类的方法（Mothod）。在方法里，又可以解析出一行行的指令。
你需要掌握两个核心的类的用法：
-
ClassReader，用来解析字节码。 -
ClassWriter，用来生成字节码。
这两个类如果配合起来用，就可以一边读入，做一定修改后再写出，从而实现对原来代码的修改。
我们先试验一下，用ClassWriter生成字节码，看看能不能生成一个跟前面示例代码中的MyClass一样的类（我们可以称呼这个类为MyClass2），里面也有一个一模一样的foo函数。：
//创建foo方法
MethodVisitor mv = cw.visitMethod(Opcodes.ACC_PUBLIC, "foo",
"(I)I", //括号中的是参数类型，括号后面的是返回值类型
null, null);
//添加参数a
mv.visitParameter("a", Opcodes.ACC_PUBLIC);
mv.visitVarInsn(Opcodes.ILOAD, 1); //iload_1
mv.visitInsn(Opcodes.ICONST_3); //iconst_3
mv.visitInsn(Opcodes.IADD); //iadd
mv.visitInsn(Opcodes.IRETURN); //ireturn
//设置操作数栈最大的帧数，以及最大的本地变量数
mv.visitMaxs(2,2);
//结束方法
mv.visitEnd();
从这个示例代码中，你会看到两个特点：
-
ClassWriter有visitClass、visitMethod这样的方法，以及ClassVisitor、MethodVistor这样的类。这是因为ClassWriter用了visitor模式来编程。你每一次调用visitXXX方法，就会创建相应的字节码对象，就像LLVM形成内存中的IR对象一样。
-
foo()方法里的指令，跟我们前面看到的字节码指令是一样的。
执行这个程序，就会生成MyClass2.class文件。
把MyClass2.class变成可读的文本格式之后，你可以看到它跟MyClass的字节码内容几乎是一样的，只有类名称不同。当然了，你还可以写一个程序调用MyClass2，验证一下它是否能够正常工作。
发现了吗？只要熟悉Java的字节码指令，在ASM的帮助下，你可以很方便地生成字节码！
既然你已经能生成字节码了，那么不如趁热打铁，把编译器前端生成的AST编译成字节码，在JVM上运行？因为这样，你就能从前端到后端，完整地实现一门基于JVM的语言了！
将AST编译成字节码
基于AST生成JVM的字节码的逻辑还是比较简单的，比生成针对物理机器的目标代码要简单得多，为什么这么说呢？ 主要有以下几个原因：
-
首先，你不用太关心指令选择的问题。针对AST中的每个运算，基本上都有唯一的字节码指令对应，你直白地翻译就可以了，不需要用到树覆盖这样的算法。
-
你也不需要关心寄存器的分配，因为JVM是使用操作数栈的；
-
指令重排序也不用考虑，因为指令的顺序是确定的，按照逆波兰表达式的顺序就可以了；
-
优化算法，你暂时也不用考虑。
当然了，我们只实现了playscript的少量特性，不过，如果在这个基础上继续完善，你就可以逐步实现一门完整的，基于JVM的语言了。
Spring与字节码生成技术
我在开篇提到，Java程序员大部分都会使用Spring。Spring的IoC（依赖反转）和AOP（面向切面编程）特性几乎是Java程序员在面试时必被问到的问题，了解Spring和字节码生成技术的关系，能让你在面试时更轻松。
Spring的AOP是基于代理（proxy）的机制实现的。在调用某个对象的方法之前，要先经过代理，在代理这儿，可以进行安全检查、记日志、支持事务等额外的功能。
There are two proxy technologies used by Spring: one is Java's dynamic proxy technology; the other is to automatically generate a proxy using cglib, which uses asm to generate bytecode.
Java's dynamic proxy technology only supports methods in the interface implemented by a certain class. If a class is not an implementation of an interface, then Spring must use cglib, thereby using bytecode generation technology to generate the bytecode of the proxy object.
Summarize
This article mainly explains bytecode generation technology. Bytecode generation technology is one of the core technologies behind the Spring framework that Java programmers are very familiar with. If you want to master this technology, you need to understand the operating principles of the Java virtual machine, the format of bytecode, and common instructions. The key points I want to emphasize are as follows:
-
There are two designs of virtual machines for running programs: one is stack-based; the other is register-based.
Stack-based virtual machines do not need to explicitly manage the addresses of operands, so instructions are shorter and instruction generation is easier. Register-based virtual machines can make better use of register resources and can also optimize the code more.
-
You need to be able to graphically imagine the operation process of the stacker in your mind, so that you can understand its principles more clearly.
-
ASM is a bytecode manipulation framework that can help you modify and generate bytecode. If you have such needs, you can use such a tool.
I also suggest that Java programmers learn more about the operating mechanism of the JVM and Java bytecode. This will better grasp the underlying mechanism of the Java language, which will be more conducive to the development of their careers.
This article is published by mdnice multi-platform