Java Agent Probe Technology

Get into the habit of writing together! This is the 5th day of my participation in the "Nuggets Daily New Plan · April Update Challenge", click to view the details of the event .

The Agent technology in Java allows us to perform non-intrusive agents, and is most commonly used in scenarios such as program debugging, hot deployment, performance diagnosis and analysis, and now the hotter distributed link tracking project Skywalking is to use probe technology to Capture logs and report the data to the OAP observation and analysis platform.

Introduction to Java Agent Technology

Java Agent Literally translated as Java Agent, also often referred to as Java Probe Technology.

Java Agent was introduced in JDK1.5 and is a technology that can dynamically modify Java bytecode. The classes in Java are compiled to form bytecodes that are executed by the JVM. Before the JVM executes these bytecodes, the information about these bytecodes is obtained, and the bytecodes are modified through a bytecode converter to complete some extra features.

Java Agent is a jar package that cannot run independently. It works by attaching to the JVM process of the target program. When starting, you only need to add the -javaagent parameter to the startup parameters of the target program to add the ClassFileTransformer bytecode converter, which is equivalent to adding an interceptor before the main method.

Java Agent function introduction

Java Agent mainly has the following functions

• The Java Agent is able to intercept and modify the Java bytecode before loading it;
• Java Agent can modify already loaded bytecode during JVM runtime;

Application Scenarios of Java Agent

• The debugging function of IDE, such as Eclipse, IntelliJ IDEA;
• Hot deployment functions, such as JRebel, XRebel, spring-loaded;
• Various online diagnostic tools, such as Btrace, Greys, and Ali's Arthas;
• Various performance analysis tools, such as Visual VM, JConsole, etc.;
• Full link performance detection tools, such as Skywalking, Pinpoint, etc.;

Java Agent Implementation Principle

在了解Java Agent的实现原理之前，需要对Java类加载机制有一个较为清晰的认知。一种是在man方法执行之前，通过premain来执行，另一种是程序运行中修改，需通过JVM中的Attach实现，Attach的实现原理是基于JVMTI。

主要是在类加载之前，进行拦截，对字节码修改

下面我们分别介绍一下这些关键术语：

• JVMTI 就是JVM Tool Interface，是 JVM 暴露出来给用户扩展使用的接口集合，JVMTI 是基于事件驱动的，JVM每执行一定的逻辑就会触发一些事件的回调接口，通过这些回调接口，用户可以自行扩展

  JVMTI是实现 Debugger、Profiler、Monitor、Thread Analyser 等工具的统一基础，在主流 Java 虚拟机中都有实现

• JVMTIAgent是一个动态库，利用JVMTI暴露出来的一些接口来干一些我们想做、但是正常情况下又做不到的事情，不过为了和普通的动态库进行区分，它一般会实现如下的一个或者多个函数：

  • Agent_OnLoad函数，如果agent是在启动时加载的，通过JVM参数设置
  • Agent_OnAttach函数，如果agent不是在启动时加载的，而是我们先attach到目标进程上，然后给对应的目标进程发送load命令来加载，则在加载过程中会调用Agent_OnAttach函数
  • Agent_OnUnload函数，在agent卸载时调用

• javaagent 依赖于instrument的JVMTIAgent（Linux下对应的动态库是libinstrument.so），还有个别名叫JPLISAgent(Java Programming Language Instrumentation Services Agent)，专门为Java语言编写的插桩服务提供支持的

• instrument 实现了Agent_OnLoad和Agent_OnAttach两方法，也就是说在使用时，agent既可以在启动时加载，也可以在运行时动态加载。其中启动时加载还可以通过类似-javaagent:jar包路径的方式来间接加载instrument agent，运行时动态加载依赖的是JVM的attach机制，通过发送load命令来加载agent

• JVM Attach 是指 JVM 提供的一种进程间通信的功能，能让一个进程传命令给另一个进程，并进行一些内部的操作，比如进行线程 dump，那么就需要执行 jstack 进行，然后把 pid 等参数传递给需要 dump 的线程来执行

Java Agent 案例

我们就以打印方法的执行时间为例，通过Java Agent 来实现。

首先我们需要构建一个精简的Maven项目，在其中构建两个Maven的子项目，一个用于实现外挂的Agent，一个用于实现测试目标程序。

我们在父应用中导入两个项目公共依赖的包

    <dependencies>
        <dependency>
            <groupId>org.javassist</groupId>
            <artifactId>javassist</artifactId>
            <version>3.28.0-GA</version>
        </dependency>
    </dependencies>
复制代码

首先我们去构建测试的目标程序

// 启动类
public class APPMain {

    public static void main(String[] args) {
        System.out.println("APP 启动！！！");
        AppInit.init();
    }
}
// 模拟的应用初始化的类
public class AppInit {

    public static void init() {
        try {
            System.out.println("APP初始化中...");
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}
复制代码

然后我们启动程序，测试是否能正常执行，程序正常执行之后，我们开始构建探针程序

探针程序中我们需要编写，改变原有class的Transformer，通过自定义的Transformer类完成输出方法执行时间的功能，

首先构检Agent程序的入口

public class RunTimeAgent {

    public static void premain(String arg, Instrumentation instrumentation) {
        System.out.println("探针启动！！！");
        System.out.println("探针传入参数：" + arg);
        instrumentation.addTransformer(new RunTimeTransformer());
    }
}
复制代码

这里每个类加载的时候都会走这个方法，我们可以通过className进行指定类的拦截，然后借助javassist这个工具，进行对Class的处理，这里的思想和反射类似，但是要比反射功能更加强大，可以动态修改字节码。

javassist是一个开源的分析、编辑和创建Java字节码的类库。

import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtMethod;

import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.IllegalClassFormatException;
import java.security.ProtectionDomain;

public class RunTimeTransformer implements ClassFileTransformer {

    private static final String INJECTED_CLASS = "com.zhj.test.init.AppInit";

    @Override
    public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
        String realClassName = className.replace("/", ".");
        if (realClassName.equals(INJECTED_CLASS)) {
            System.out.println("拦截到的类名：" + realClassName);
            CtClass ctClass;
            try {
                // 使用javassist,获取字节码类
                ClassPool classPool = ClassPool.getDefault();
                ctClass = classPool.get(realClassName);

                // 得到该类所有的方法实例，也可选择方法，进行增强
                CtMethod[] declaredMethods = ctClass.getDeclaredMethods();
                for (CtMethod method : declaredMethods) {
                    System.out.println(method.getName() + "方法被拦截");
                    method.addLocalVariable("time", CtClass.longType);
                    method.insertBefore("System.out.println(\"---开始执行---\");");
                    method.insertBefore("time = System.currentTimeMillis();");
                    method.insertAfter("System.out.println(\"---结束执行---\");");
                    method.insertAfter("System.out.println(\"运行耗时: \" + (System.currentTimeMillis() - time));");
                }
                return ctClass.toBytecode();
            } catch (Throwable e) { //这里要用Throwable，不要用Exception
                System.out.println(e.getMessage());
                e.printStackTrace();
            }
        }
        return classfileBuffer;
    }
}
复制代码

我们需要在Maven中配置，编译打包的插件，这样我们就可以很轻松的借助Maven生成Agent的jar包

<build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.5.1</version>
                <!-- 指定maven编译的jdk版本。若不指定,maven3默认用jdk 1.5 maven2默认用jdk1.3 -->
                <configuration>
                    <source>8</source>
                    <target>8</target>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>3.2.0</version>
                <configuration>
                    <archive>
                        <!--自动添加META-INF/MANIFEST.MF -->
                        <manifest>
                            <addClasspath>true</addClasspath>
                        </manifest>
                        <manifestEntries>
                            <Menifest-Version>1.0</Menifest-Version>
                            <Premain-Class>com.zhj.agent.RunTimeAgent</Premain-Class>
                            <Can-Redefine-Classes>true</Can-Redefine-Classes>
                            <Can-Retransform-Classes>true</Can-Retransform-Classes>
                        </manifestEntries>
                    </archive>
                </configuration>
            </plugin>
        </plugins>
    </build>
复制代码

Otherwise, we need to create the META-INF/MANIFEST.MF file under resources. The content of the file is as follows. We can see that this is consistent with the configuration in Maven. Then, by configuring the compiler, package it into a jar package with the help of the compiler, which needs to be specified. The document

Manifest-Version: 1.0
Premain-Class: com.zhj.agent.RunTimeAgent
Can-Redefine-Classes: true
Can-Retransform-Classes: true

复制代码

Notice file MANIFEST.MF parameter description:

• Manifest-Version

  file version

• Premain-Class

  The class containing the premain method (the full path name of the class) the main method runs the proxy before

• Agent-Class

  The class containing the agentmain method (full path name of the class) can modify the class structure after main starts

• Boot-Class-Path

  Sets the list of paths searched by the bootstrap class loader. These paths are searched by the bootstrap class loader after the platform-specific mechanism for finding classes fails. Search paths in the order listed. The paths in the list are separated by one or more spaces. (optional)

• Can-Redefine-Classes true

  Indicates the classes required to be able to redefine this proxy, the default value is false (optional)

• Can-Retransform-Classes true

  Indicates the class required to be able to retransform this proxy, the default value is false (optional)

• Can-Set-Native-Method-Prefix true

  Indicates that the native method prefix required by this proxy can be set, the default value is false (optional)

• ...

Finally, generate the jar package of the Agent through Maven, then modify the launcher of the test target program, and add JVM parameters.

参数示例：-javaagent:F:\code\myCode\agent-test\runtime-agent\target\runtime-agent-1.0-SNAPSHOT.jar=hello

final effect:

This completes a non-intrusive proxy.