Java Hot Deployment refers to updating and reloading part of the code or resources of a Java application at runtime without stopping the entire application. This capability is useful for rapid iteration and troubleshooting during development and debugging.

In the past, Java's hot deployment capabilities were relatively weak and usually required the use of specific frameworks or tools. However, as time goes by, the demand for hot deployment among Java developers continues to increase, and many new solutions and tools have emerged to make Java hot deployment easier and more common.

Here are some of the Java hot deployment technologies and tools currently available:

JRebel : JRebel is a commercial tool that enables hot deployment of Java code and resources without restarting the application. It implements hot deployment capabilities by using class loader technology and bytecode transformation. JRebel is widely used in development environments and debugging processes, but requires purchasing a license to use it.
Spring Boot DevTools : Spring Boot DevTools is a development toolkit that provides many convenient features during development, including hot deployment. It can monitor code and resource changes during development and automatically reload the corresponding parts to speed up the development cycle.
DCEVM : Dynamic Code Evolution VM (DCEVM) is a Java virtual machine patch that extends the capabilities of the JVM to make it possible to modify class definitions at runtime without restarting. DCEVM is used in conjunction with HotSwap technology to achieve class-level hot deployment.
Integration of JRebel and DCEVM : JRebel and DCEVM can be used together to achieve more powerful hot deployment capabilities. This integration provides broader hot deployment coverage and provides more advanced debugging and development capabilities.

Although the ability to hot deploy Java has improved over the past few years, there are still some limitations and challenges. For example, certain types of code changes, such as changes to a class's inheritance structure, may require an application restart to take effect. Additionally, some hot deployment solutions may require specific application architectures and environments.

Overall, the current state of Java hot deployment has improved significantly, allowing developers to develop and debug more quickly and efficiently. However, the specific choice depends on the needs of the application and the preferences of the development team.

(2) Analysis of technical implementation difficulty

The technical difficulties of Java hot deployment mainly involve the following aspects:

Management of class loaders and class definitions : Java hot deployment relies on the mechanism of class loaders to load and reload class definitions. During the hot deployment process, it is necessary to correctly manage the relationship between class loaders and classes to ensure that new class definitions can be loaded correctly and replace old definitions, while avoiding problems such as memory leaks and class loading conflicts.
Bytecode transformation and dynamic modification : During hot deployment, bytecode needs to be transformed and modified to apply new code and resources to the running application. This involves parsing, modifying and reloading the bytecode, while ensuring the correctness and consistency of the modified bytecode to avoid abnormal behavior or crashes of the application.
Resource management and update : In addition to class definitions, hot deployment also needs to consider the update and management of other resources in the application, such as configuration files, template files, etc. Ensuring that these resources are loaded and applied correctly, and updated in sync with the code, is a challenge.
State and data consistency : Hot deployment may disrupt ongoing business logic and data processing. When reloading code and resources, you need to consider how to maintain application state and data consistency to avoid data loss, business errors, or inconsistencies.
Adaptation to specific frameworks and environments : Different Java frameworks and application servers may support hot deployment to varying degrees and in different ways. When implementing hot deployment, it is necessary to adapt and adjust the specific framework and environment to ensure the stability and compatibility of the hot deployment function.

These technical difficulties need to be considered comprehensively and solved and implemented in hot deployment tools and frameworks. Different tools and frameworks may adopt different methods and strategies to address these challenges to provide reliable and efficient hot deployment capabilities.

At present, the Java series officially introduced by Meituan | The implementation practice of remote hot deployment in Meituan - the hot deployment plug-in Sonic of Meituan technical team came into being.

(3) Analysis of its necessity

Generally speaking, during the development process of most enterprises, engineers need to release services multiple times a day while testing and debugging corresponding requirements. Most applications take about 3 to 10 minutes each time. In addition, the single time for building, deploying, and image processing is concentrated in 20 minutes. ~45 minutes. The deployment is frequent and time-consuming, which seriously affects the efficiency of system rollout. Hot deployment is an important necessity in Java development. Here is some analysis:

Improve development efficiency: Hot deployment allows developers to update code and resources without restarting the entire application. This means developers can make code modifications and experiments faster, iterating and debugging applications quickly. Hot deployment can reduce the development cycle and improve development efficiency.
Enables rapid feedback and debugging: Hot deployment enables developers to immediately see the results of changes they make to their code, without the need to rebuild and restart the application. This rapid feedback mechanism can help developers quickly debug and fix problems, improving development efficiency and quality.
Reduce system downtime: Traditional deployment methods often require stopping the application, redeploying, and starting, which results in system downtime. Hot deployment avoids this downtime, allowing parts of the code and resources to be updated and reloaded at runtime, enabling seamless application updates and upgrades.
Improved user experience: Hot deployment enables application updates without interrupting ongoing business processes. This means users can get the latest features and fixes while using the app, improving user experience and satisfaction.
Supports dynamic configuration and flexibility: Hot deployment enables dynamic modification of an application's configuration and behavior without the need to restart the application. This provides greater flexibility and configurability, allowing real-time adjustments and configuration as needed.

Although hot deployment has many advantages and necessity in Java development, there are also some potential challenges and limitations to be aware of. For example, some types of code changes may require an application restart to take effect, and hot deployment may introduce some runtime complexity and require appropriate tooling and framework support. Therefore, when selecting and using hot deployment technology, factors such as project requirements, the technical level of the development team, and tool costs need to be comprehensively considered.

If hot deployment conditions are met, it will help programmers play a great role in development and self-test joint debugging. The following is a comparison chart after Meituan went online and used Sonic.

Develop self-test scenarios

Joint debugging scene

2. Enter Meituan’s Java remote hot deployment practice

(1) Sonic analysis

Sonic is an IDEA plug-in developed and designed internally by Meituan. It aims to assist remote/local hot deployment through low-code development, solve efficiency problems in coding, single test writing and execution, self-test and joint debugging, etc., and improve developers' coding productivity. Output efficiency. Statistics show that developers spend about 35% of their daily time on coding output. If you want to improve R&D efficiency, you must either expand the time proportion of coding output or improve the output efficiency of the coding stage. Sonic focuses on improving the output efficiency of the coding stage.

Currently, using Sonic hot deployment can solve most of the code duplication building problems. Sonic allows users to write code locally and deploy it to a remote environment with one click, modify the code, deploy, jointly debug requests, view logs, and repeat the cycle. If the code modification time is not considered, a cycle usually takes 20 to 35 minutes. Using Sonic can shorten the entire time to 5 to 10 seconds, and can provide developers with an efficient and immersive development experience. In actual coding work, multiple file modifications are commonplace, and Sonic's hot deployment capabilities for multiple files are particularly outstanding. It can perform remote hot deployment of multiple files in batches through dependency analysis and other means, and supports Spring Bean Class, ordinary Class, Spring Mixed hot deployment of XML, MyBatis XML and other types of files.

(2) Comparison of existing products in the industry

JRebel supports many third-party plug-ins and has a huge ecosystem, but it does not support domestic plug-ins, such as FastJson, etc. At the same time, it also has limitations in remote hot deployment configuration. The middleware within the company requires personalized development, and it is commercial software. The cost of use is higher .

(2) Practical experience in the implementation and promotion of Sonic remote hot deployment

For the promotion of technical products, especially products used in the development and testing stages, since they are far away from the online environment, whether the driving force, execution power, and product function closed loop can be done well will determine whether the product can be launched within the enterprise and gain widespread popularity. An important part recognized by most people. In addition, because many developers have gradually formed "solidified actions" during the development and testing stages, how to change the behavior of these users and let them embrace new products is also one of the difficult challenges Sonic faces.

The Sonic team has implemented the above four principles starting from active communication, zero-cost (or extremely low-cost) quick access, automated scripts, automatic product diagnosis, and collection of feedback.

3. Overall design plan

(1) Sonic structure

Sonic plug-in consists of 4 parts, including script side, plug-in side, Agent side, and Sonic server side. The script side is responsible for automatically building Sonic startup parameters, service startup and other integration work; the IDEA plug-in side integration environment provides developers with more convenient hot deployment services; the Agent side is responsible for implementing the hot deployment function when the project is started; the server side is responsible for collecting hot deployment Information, failure reporting and other statistical work. As shown below:

(2) Agent analysis

Agent technical analysis

Agent in software development refers to a tool or library that can monitor, modify, and extend an application while it is running. Agent can implement the implantation and operation of applications through bytecode injection, dynamic proxy, code conversion and other technologies .

Agent technology plays an important role and is widely used in Java development. The following is an analysis of Agent technology:

Dynamic monitoring and diagnosis : Agent can monitor and collect various indicators and information while the application is running, such as performance indicators, memory usage, method calls, etc. This is useful for application performance analysis, problem diagnosis, and optimization. Agent can capture key events and generate corresponding logs or reports through customized logic and rules.
Code enhancement and correction : Agent can implement code enhancement and correction to the application program by modifying the bytecode at runtime. For example, you can insert custom logic before and after method calls to modify or verify parameters. This capability allows applications to be customized and enhanced without modifying the source code.
Dynamic loading and unloading : Agent can dynamically load and unload class definitions through a customized class loader mechanism. This is useful for implementing plug-in architecture, modular development and dynamic expansion. Agent can dynamically load new classes and resources as needed and unload them when not needed, thereby achieving application dynamism and flexibility .
Security check and permission control : Agent can perform security check and permission control on the code while the application is running. For example, you can intercept and authenticate calls to sensitive methods, or perform permission checks on access to restricted resources. This helps enhance application security and controllability.
AOP (aspect-oriented programming) support : Agent can be used in conjunction with the AOP framework to manage and operate the cross-cutting concerns of the application. Through Agent, applications can be programmed at runtime to implement common functions such as logging, performance monitoring, and transaction management.

It should be noted that Agent technology needs to be used with caution because runtime implantation and manipulation of applications may introduce performance overhead and complexity. At the same time, for some security-sensitive environments, the use of Agent and permission control need to be carefully evaluated to ensure the security and stability of the system.

Some well-known Java Agent tools include Byte Buddy, AspectJ, Java Agent API, etc. These tools provide rich functions and flexible extensibility, allowing developers to customize and use Agent technology as needed.

Overview of core principles

The core principles of Agent involve bytecode operations and class loader mechanisms. The following is an analysis of the core principles of Agent:

Bytecode operation : Agent operates the bytecode of the application to enhance and correct the application. Bytecode operations can include inserting new instructions, modifying existing instructions, deleting instructions, etc. This kind of operation can dynamically change the behavior of the application at runtime.
Class loader mechanism : Agent uses Java's class loader mechanism to load and define agent classes. It loads the agent class generated by Agent through a custom class loader and defines it as a new class definition. This allows the Agent to dynamically create and load classes at runtime without modifying the original class definition.
Dynamic proxy : Agent often uses dynamic proxy technology to implement the implantation and operation of applications. Dynamic proxies can intercept method calls to the original class by creating proxy classes and proxy objects, and insert custom logic before and after the calls. This allows application enhancements and corrections to be made without modifying the original classes.
JVM Attach mechanism : Agent usually uses JVM Attach mechanism to attach itself to the target Java process. The JVM Attach mechanism allows the Agent to access the interior of the target process and communicate with the Java virtual machine. This enables the Agent to inject its own code into the target process at runtime and interact with the application.
Java Agent API: Java provides the Java Agent API, which defines a set of interfaces and classes for writing and managing Java Agents. Through the Java Agent API, the Agent can register callback methods and perform corresponding processing when events such as application loading, initialization, conversion, etc. occur .

The core principle of Agent is to achieve dynamic enhancement and correction of applications by operating bytecode and class loaders at runtime. It allows developers to intercept and modify application behavior through customized logic and rules, thereby customizing and extending applications. This capability plays an important role in monitoring, diagnosis, security control, AOP and other scenarios.

Instrument tool analysis

Instrument is a tool class in the Java language, located in the java.lang.instrument package. It provides a set of tools and methods for converting the bytecode of Java applications at runtime. The Instrumentation API allows developers to modify bytecode, define new classes, and redefine loaded classes during the class loading process.

Instrument is mainly used to develop Java Agent, which provides the Agent with underlying access and operation capabilities to Java applications. Through the Instrumentation API, developers can perform custom bytecode transformations and operations during the life cycle of a Java application. The main functions of Instrument include:

Registering a converter (Transformer) : Instrument provides the addTransformer method for registering a custom class converter (ClassFileTransformer). A class converter is a class that implements the ClassFileTransformer interface and is used to convert the bytecode of a class during the class loading process.
Transform the bytecode of a class : The transform method of the ClassFileTransformer interface allows developers to modify the bytecode of a class. By implementing this method, you can perform customized operations on the bytecode of the class when the class is loaded, such as inserting new instructions, modifying existing instructions, etc.
Redefine classes : Instrument provides the redefineClasses method, allowing developers to redefine loaded classes. Redefinition and replacement of classes can be achieved by replacing the definition of a loaded class by providing new class bytecode.
Get the definition of a class : The getAllLoadedClasses method of Instrument can get the definition of all classes currently loaded. By traversing loaded classes, developers can obtain the class name, bytecode, and other information.
Dynamically add proxy classes : Instrument's appendToBootstrapClassLoaderSearch method can add proxy classes to the search path of the boot class loader. In this way, the proxy class can be loaded by the bootstrap class loader and thus function throughout the application.

The use of Instrument is usually implemented in combination with Java Agent. Java Agent uses the Instrumentation API to dynamically implant and operate Java applications. Through an instance of Instrumentation, Agent can register a class converter and convert and redefine classes, thereby customizing and enhancing the application.

Instrumentation analysis

Instrumentation is an API provided by Java and is located in the java.lang.instrument package. It provides a set of tools and methods for converting bytecode while Java applications are running. The Instrumentation API allows developers to modify bytecode, define new classes, and redefine loaded classes during the class loading process.

Agent uses the Instrumentation API to implement dynamic implantation and operation of applications. Through the combination of Agent and Instrumentation API, classes can be converted, modified and enhanced during the application loading phase or runtime. Specifically, the Agent can register a ClassFileTransformer through the Instrumentation API and implement its transform method to convert the bytecode of the class.

When the Java application starts, the Agent attaches itself to the target process through the JVM Attach mechanism and obtains the Instrumentation instance in the premain method or agentmain method. Then, the Agent uses the Instrumentation instance to register a custom ClassFileTransformer and transform the class during the class loading process.

The Instrumentation API provides the following functions to support Agent implementation:

Register ClassFileTransformer : Agent uses the addTransformer method of the Instrumentation instance to register a custom ClassFileTransformer to transform classes during the class loading process.
Transform the bytecode of the class : The transform method of the ClassFileTransformer interface allows the Agent to modify and transform the bytecode of the class. Agent can obtain the bytecode of the class in this method and perform customized operations on it, such as inserting new instructions, modifying existing instructions, etc.
Redefine classes : The redefineClasses method of the Instrumentation API allows the Agent to redefine loaded classes. Agent can redefine and replace classes by providing new class bytecode to replace the definition of loaded classes.

The combination of Instrumentation and Agent allows developers to dynamically implant and operate Java applications at runtime to customize, enhance and modify applications. Through the Instrumentation API, Agents can modify the behavior of applications in a non-intrusive manner without modifying the source code.

Commonly used APIs of the Instrumentation class are as follows:

public interface Instrumentation {

    //增加一个Class 文件的转换器，转换器用于改变 Class 二进制流的数据，参数 canRetransform 设置是否允许重新转换。
    void addTransformer(ClassFileTransformer transformer, boolean canRetransform);

    //在类加载之前，重新定义 Class 文件，ClassDefinition 表示对一个类新的定义，
    //如果在类加载之后，需要使用 retransformClasses 方法重新定义。addTransformer方法配置之后，后续的类加载都会被Transformer拦截。
    //对于已经加载过的类，可以执行retransformClasses来重新触发这个Transformer的拦截。类加载的字节码被修改后，除非再次被retransform，否则不会恢复。
    void addTransformer(ClassFileTransformer transformer);

    //删除一个类转换器
    boolean removeTransformer(ClassFileTransformer transformer);
    
    //是否允许对class retransform
    boolean isRetransformClassesSupported();

    //在类加载之后，重新定义 Class。这个很重要，该方法是1.6 之后加入的，事实上，该方法是 update 了一个类。
    void retransformClasses(Class<?>... classes) throws UnmodifiableClassException;
   
    //是否允许对class重新定义
    boolean isRedefineClassesSupported();

    //此方法用于替换类的定义，而不引用现有的类文件字节，就像从源代码重新编译以进行修复和继续调试时所做的那样。
    //在要转换现有类文件字节的地方（例如在字节码插装中），应该使用retransformClasses。
    //该方法可以修改方法体、常量池和属性值，但不能新增、删除、重命名属性或方法，也不能修改方法的签名
    void redefineClasses(ClassDefinition... definitions) throws  ClassNotFoundException, UnmodifiableClassException;

    //获取已经被JVM加载的class，有className可能重复（可能存在多个classloader）
    @SuppressWarnings("rawtypes")
    Class[] getAllLoadedClasses();
}

Analysis of the process of loading Instrument Agent at startup and runtime

(3) The “love and death” between JVM and HotSwap

Improvements are constantly being made in the HotSwap JVM surrounding Method Body.

Starting from version 1.4, JPDA introduces the HotSwap mechanism (JPDA Enhancements) to realize the dynamics of Method Body during debugging. You can refer to the document: enhancements1.4 .
Starting from version 1.5, the Premain mode of java.lang.instrument (Java Platform SE 8) implemented by JVMTI is used to realize the dynamics of the Agent mode (the Agent is specified when the JVM starts) . You can refer to the document: package-summary .

Version 1.6 adds the Agentmain mode to achieve runtime dynamics (binding to a specific VM through The Attach API) . You can refer to the document: package-summary . The basic implementation is to update method and body-level bytecode through JVMTI's retransformClass/redefineClass. ASM and CGLib basically make dynamics around these. However, there has been no action for HotSwap for Class (such as adding methods, adding fields, modifying inheritance relationships, etc.) to Class. Why is this happening? Because the complexity is too high and there is no high return.

Analysis of HotSwap mechanism (JPDA Enhancements)

The HotSwap mechanism is a functional enhancement provided by the Java Platform Debugging Architecture (JPDA). JPDA is part of the Java platform and is used to support developers to perform interactive debugging operations when debugging applications.

The HotSwap mechanism allows modifications to loaded classes while the application is running without restarting the application. This is useful for rapid iteration and debugging during development, as developers can make changes to code at runtime and immediately see the effects of modifications without having to stop and restart the application.

The main features and principles of the HotSwap mechanism include the following aspects:

Dynamically update bytecode : The HotSwap mechanism implements class updates by replacing the bytecode of the class at runtime. Developers can use a debugger or other JPDA-enabled tool to edit the bytecode of a class and send the modified bytecode to the running virtual machine.
Code hot replacement : The HotSwap mechanism supports modification of the method body to achieve hot code replacement. Developers can change the implementation code of a method at runtime and then reload the method so that the new code takes effect immediately.
Limitations and constraints : The HotSwap mechanism has some limitations and constraints. For example, you can only replace the method body, but not add new fields or methods. In addition, code changes in some scenarios may require reinitialization or reconnection of the application.
JPDA support : The HotSwap mechanism is provided through the Java Platform Debug Architecture (JPDA). JPDA is a set of APIs and protocols for debugging and monitoring Java applications. Through the enhanced functions of JPDA, developers can use the HotSwap mechanism to achieve the need to update code at runtime.

In summary, the HotSwap mechanism is a functional enhancement of JPDA that allows developers to modify loaded classes at runtime to achieve hot replacement of code. It provides developers with a way to quickly iterate and debug code, saving time on restarting applications and improving development efficiency.

JVMTI analysis

JVMTI (Java Virtual Machine Tool Interface) is a Java Virtual Machine Tool Interface that allows developers to develop and connect tools to the Java Virtual Machine in order to monitor, debug, and analyze the execution of Java applications.

The following is an analysis of JVMTI:

Functional Overview: JVMTI provides a set of APIs that allow developers to write tools to monitor and control the behavior of the Java Virtual Machine. It allows developers to obtain information about classes, objects, threads, stacks, memory, etc. during application execution, and dynamically modify and control it.
Monitoring and debugging functions: JVMTI allows developers to monitor events occurring in the Java virtual machine through the event notification mechanism, such as class loading, method invocation, exception throwing, etc. Developers can register event listeners to perform appropriate actions when events occur. In addition, JVMTI also provides the ability to monitor and analyze threads, stacks, memory, etc.
Dynamic modification function: JVMTI allows developers to modify Java classes while the application is running. Developers can use the functions provided by JVMTI to redefine classes, modify method bodies, add fields, etc. This ability to dynamically modify is very useful for some specific application scenarios (such as code hot replacement).
Performance analysis functions: JVMTI also provides some performance analysis tools, such as memory analysis, thread analysis and CPU analysis. Developers can use these tools to identify performance bottlenecks, memory leaks, and other issues and optimize the performance of Java applications.
Embedded and remote connections: JVMTI supports embedding tool agents within the Java virtual machine and also supports remote connections to interact with applications running on remote Java virtual machines. This allows developers to remotely monitor and debug distributed Java applications.

To summarize, JVMTI is the Java Virtual Machine Tools Interface, which provides a set of APIs and functions that allow developers to develop and connect tools to the Java Virtual Machine. It provides the ability to monitor, debug, analyze and dynamically modify Java applications, providing developers with rich tools and interfaces to deeply understand and control the execution of Java applications.

Agentmain implements runtime dynamic analysis

Agentmain is a method in the Java Agent mechanism that is used to dynamically load and initialize Java Agent when the application is running. Through Agentmain, developers can load the Java Agent into the running Java virtual machine after the application is started, and perform dynamic analysis of the application at runtime.

The implementation principles and steps of Agentmain are as follows:

Writing Java Agent: First, developers need to write a Java Agent, which contains the functions and logic required for implementation. Java Agent can use the Instrumentation API to monitor and modify running applications.
Launch the application: The developer then needs to launch the target application, either through the command line or other means. When starting the application, you need to specify the -javaagent option in the JVM startup parameters and pass the Java Agent's JAR file path as a parameter to the -agentmain parameter.
Load Agentmain: After the application starts, the Java virtual machine automatically loads and initializes Agentmain. The premain method in Agentmain will be called to initialize the Java Agent. In the premain method, developers can obtain the Instrumentation instance and register the required monitors and converters.
Runtime dynamic analysis: Once Agentmain is successfully loaded and initialized, developers can use the Instrumentation API to perform dynamic analysis of the running application. For example, you can add method interceptors, collect performance metrics, log, etc. Through the Instrumentation API, you can modify the class definition, replace the method body, or add additional bytecode logic.

In summary, Agentmain is an implementation method in the Java Agent mechanism, which is used to dynamically load and initialize Java Agent when the application is running. Through Agentmain, developers can use the Instrumentation API to dynamically analyze and modify running applications. This provides developers with a flexible and powerful tool for implementing a variety of monitoring, analysis, and optimization capabilities within the application's runtime environment.

(4) Sonic solves the limitations of Instrumentation

Due to JVM limitations, both JDK 7 and JDK 8 do not allow changes to the class structure, such as adding fields, adding methods, and modifying the parent class of the class, etc. This is fatal for Spring projects. For example, a developer wants to modify a Spring Bean and add an @Autowired field. There are many such scenarios in practical applications, so Sonic's support for such scenarios is essential.

So, how exactly is it done? Here I would like to mention the "famous" Dcevm. Dcevm (DynamicCode Evolution Virtual Machine) is a patch (strictly speaking, a modification) of Java Hostspot that allows (but not unlimited) modification of loaded class files in the running environment. The current virtual machine only allows modification of the method body (Method, Body), while Decvm can add, delete class attributes, methods, and even change the parent class of a class. Dcevm is an open source project that complies with the GPL 2.0 agreement. For more introduction to Dcevm, you can refer to: Wuerthinger10a and GitHub Decvm .

It is worth mentioning that within Meituan, Sonic has opened HULK for the installation of Dcevm, which can be completed by integrating the release image (local hot deployment can be combined with the plug-in function to achieve a one-click installation hot deployment environment).

Dcevm analysis

DCEVM (Dynamic Code Evolution VM) is an enhanced tool for Java Virtual Machine (JVM) that allows Java class files to be modified and reloaded at runtime, thereby enabling hot deployment and rapid development capabilities.

The following is an analysis of DCEVM:

Hot deployment and rapid development : The main goal of DCEVM is to provide a convenient way for hot deployment and rapid development. In the traditional Java development process, modifying Java class files usually requires recompiling and restarting the application. DCEVM allows dynamically loading and replacing modified Java class files without restarting the application, achieving immediate effect.
Enhanced class loading mechanism : DCEVM enhances the JVM's class loading mechanism to enable it to dynamically load and unload class files at runtime. It provides a more flexible class loader implementation than the standard JVM, capable of loading new class definitions and replacing the definitions of loaded classes while the application is running.
Fast code hot replacement : DCEVM also provides fast code hot replacement (HotSwap) function. It allows developers to modify the method body of a loaded class while the application is running, with immediate effect. This rapid code hot replacement can greatly improve development efficiency and reduce development and debugging time.
Compatibility : DCEVM is compatible with standard JVMs and can be used with multiple JVM implementations. It can be integrated with OpenJDK, Oracle JDK and other JVMs to provide application enhancements.
Deployment and configuration : Using DCEVM requires configuring it as an alternative version of the JVM and integrating it with the target application. DCEVM provides corresponding binary distribution and installation tools, making deployment and configuration relatively simple.

In summary, DCEVM is a tool that enhances the JVM. It enables developers to dynamically load and modify Java class files at runtime by providing hot deployment and rapid development capabilities. It can realize hot code replacement with immediate effect, improving development efficiency and flexibility. DCEVM is compatible with standard JVMs and can be integrated with multiple JVMs, providing developers with a convenient and powerful tool to accelerate the development and debugging process of Java applications.

4. Analysis of Sonic Hot Deployment Technology

(1) Sonic overall architecture model

(2) Sonic function transfer

Sonic monitors local file changes through NIO and triggers file change events, such as Class addition, Class modification, Spring Bean reloading and other event processes. The following figure shows the life cycle of a single file during a hot deployment:

(3) File monitoring

Sonic will first predefine two directories, local and /var/tmp/sonic/extraClasspathremote /var/tmp/sonic/classes. extraClasspath is Sonic's customized expanded Classpath URL, and classes is the directory that Sonic monitors. When files are changed, they are deployed to remote/local locations through the IDEA plug-in, triggering the Agent's monitoring directory to continue the following hot reloading logic:

Why doesn't Sonic directly replace the resource files under the user's ClassPath?

Because considering that the API projects of the business side WAR package, Spring Boot, Tomcat project, Jetty project, etc. are all started with JAR packages, the user's Class file cannot be directly modified. Even if the user project can be modified, directly operating the user's Class will bring about a series of security issues.

Therefore, Sonic uses expanded ClassPath URL paths to modify and add files. And there is a scenario where multiple business-side projects introduce the same JAR package and configure the XML and annotations of MyBatis in the JAR. In this case, Sonic has no way to directly modify the source files in the JAR package. By expanding the path, it can modify a certain file and XML in the JAR package without paying attention to the JAR package. In the same way, this method can be used to hot replace the entire JAR package. Let’s briefly introduce Sonic’s core listener, as shown in the figure below:

(4) JVM Class Reload

JVM's bytecode batch reloading logic generates ClassDefinition definitions and instrumentation.redefineClasses (definitions) through the new bytecode binary stream and the old Class object to trigger JVM reloading. After overloading, Spring plug-in registration will be triggered during initialization. Transfrom. Next, we briefly explain how Spring is overloaded.

How to ensure that the newly added class Sonic can be loaded into the Classloader context? Since the project is executed remotely, the running environment is complex. It may be started in JAR package mode (Spring Boot), it may be an ordinary project, or it may be a War Web project. For such situations, Sonic has made a layer of Classloader URL expansion.

User ClassLoader is the collective name for the framework's customized ClassLoader. For example, the Jetty project is WebAppclassLoader. The Urlclasspath is the lib file of the current project. For example, the Spring Boot project also loads CLass from the current project BOOT-INF/lib/ path, etc. The custom locations of different frameworks are slightly different. Therefore, for this kind of situation, the Agent must get the user's custom Classloader. If it is started in a conventional way, such as an ordinary Spring XML project, published with the help of Plus (Meituan's internal service publishing platform), there is no custom Classloader for this type, which is the default AppClassLoader, so the Agent uses bytecode enhancement to obtain the real user Classloader during the user project startup process.

After finding the sub-Classloader used by the user, obtain the element Classpath in the Classloader through reflection. The URL in the ClassPath is all the runtime Class environments required when the current project loads Class, and includes third-party JAR package dependencies, etc.

Sonic obtains the URL array and adds Sonic's customized extended Classpath directory to the first place in the URL array. In this way, when a new Class is added, Sonic only needs to copy the Class file to the package directory corresponding to the extended Classpath. When there are other Beans When relying on a newly added Class, the class file will be searched from the current directory.

Why not just strengthen Appclassloader? But to strengthen the framework's custom Classloader?

Consider such a scenario. There is ClassA in the framework's custom class loader. At this time, the user's new ClassB needs to be hot loaded. Class B has a reference relationship with A. If AppClassLoader is enhanced, ClassLoader is initialized when B instance is initialized. loadclass first loads the bytecode of ClassB from UserClassLoader. Relying on the principle of parental delegation, B is loaded by Appclassloader. Because B depends on class A, the current AppClassLoader loading B must not be able to load it. At this time, a ClassNotFoundException exception will be thrown. Therefore, when expanding the class loader, the top-level class loader must be expanded, so as to achieve the effect the user wants.

(5) Spring Bean overloading

During the Spring Bean Reload process, the Bean destruction and restart process, the main content is shown in the following figure:

How to load Spring Beans

First, when modifying Java Class D, scan and verify whether the currently modified Bean is a Spring Bean (annotation verification) through Spring ClasspathScan, and then trigger the destruction process (BeanDefinitionRegistry.removeBeanDefinition). This method will remove Bean D and dependencies in the current Spring context. Bean C of Spring Bean D is also destroyed, but the scope is only in the current Spring context. If C is dependent on Bean B in the subcontext, the dependency relationship in the subcontext cannot be updated. When there is a system request, Bean C associated in Bean B is still the object before hot deployment, so hot deployment fails.

Therefore, during the Spring initialization process, the corresponding relationship between the parent and child contexts needs to be maintained. When the child context changes, if the change scope involves Bean B, the dependencies in the child context need to be updated again. When there are multiple contexts associated, multiple contexts need to be maintained. environment, and the current context environment entry requires Reload. The entrances here refer to: Spring MVC Controller, Mthrift and Pigeon. Different Reload strategies are used for different traffic entrances. The main operations of the RPC framework entrance are to unbind the registration center, re-register, reload the startup process, etc. For the Spring MVC Controller, the main operations are to unbind and register the URL Mapping to implement changes in the traffic entry class.

(6) Spring XML overloading

When users modify/add Spring XML, all beans in the XML need to be reloaded.

After reloading, destroy Spring and restart it. It should be noted that the XML modification method has undergone major changes, which may involve global AOP configuration and pre- and post-processor related content. The scope of impact is global, so currently only the addition of ordinary XML Bean tags is allowed. /Modification, other abilities will be gradually released as appropriate.

(7) MyBatis hot deployment

The main processing process of Spring MyBatis hot deployment is to obtain all Configuration paths during startup, maintain its correspondence with Spring Context, match Configuration when hot deploying Class and XML, and then reload Configuration to achieve the purpose of hot deployment.

5. Sonic application

(1) Overview of hot deployment functions

In addition to the reloading process of Spring Bean, Spring MVC, and MyBatis, Sonic also supports other commonly used development frameworks. Rich framework support and compatibility are the cornerstone of Sonic. Here are some commonly used third-party frameworks supported by Sonic:

So far, Sonic has supported hot loading of most commonly used third-party frameworks, and there is almost no need to restart services for routine business development. And the success rate within Meituan has reached over 99.9%, making it truly possible for hot deployment to replace conventional deployment and construction.

(2) IDE plug-in integration

Sonic also provides a powerful IDEA plug-in, allowing users to conduct immersive development and remote hot deployment becomes more convenient.

Reference links

Java series | Implementation practice of remote hot deployment in Meituan - Meituan technical team : main learning materials
"Java Agent and Instrumentation Tutorial by Baeldung" - Baeldung's well-known Java developer website provides tutorials and sample codes about Java Agent and Instrumentation. You can search for relevant tutorials on the Baeldung website, such as "Java Agent tutorial Baeldung".
"Oracle Official Documentation" - Oracle provides official documentation for Java SE. Detailed instructions and examples for the Instrumentation API and Java Agent can be found on Oracle's official website. Visit Oracle's official website and search for "Java SE documentation" to enter the official documentation page.
"Byte Buddy Official Documentation" - Byte Buddy's popular Java bytecode manipulation library, which is widely used to develop Java Agents. You can visit Byte Buddy’s official website and find detailed guides and examples on using Byte Buddy and the Instrumentation API on the website.

Practical learning and analysis of Java remote hot deployment

1. Analysis of hot deployment status and necessity

(1) Hot deployment definition and current situation analysis