The plug-in development model is being widely applied in many programming languages or technical frameworks, such as the familiar jenkins, the docker visual management platform rancher, and the editor idea and vscode used for daily coding, etc., which can be seen everywhere with hot-plug Functional plug-ins make the system feel like wings, greatly improving the scalability and scalability of the system, and also expanding the overall use value of the system, so why use plug-ins?

1.1 Benefits of using plugins

1.1.1 Module decoupling

There are many ways to achieve decoupling between service modules, but for plug-ins, the degree of decoupling seems to be higher, more flexible, and better customizable and personalized.

For example, design patterns can be used in the code to choose which method to use to send SMS to customers who have placed an order. The problem is that each SMS service provider may not guarantee that the SMS can be sent successfully under any circumstances. What should I do? At this time, the design mode can't help you solve this problem. If you use a customized plug-in method, combined with external configuration parameters, assuming that a certain SMS in the system cannot be sent, you can use the plug-in to dynamically implant and switch to a different The manufacturer sent a text message.

1.1.2 Improve scalability and openness

Taking spring as an example, the reason why it has such a wide ecology is inseparable from its various built-in extensible plug-in mechanisms. Just imagine why you can easily connect to other middleware after using the spring framework. That is the spring framework. Provides many extension points based on plug-in.

The plug-in mechanism improves the scalability of the system, thereby enriching the surrounding application ecology of the system.

1.1.3 Facilitate third-party access

After having a plug-in, if a third-party application or system wants to connect to its own system, it can directly complete a set of implementations suitable for its own business based on the plug-in interface reserved by the system, and it is very intrusive to its own system. It can even be implemented based on Hot loading of configuration parameters, convenient and flexible, out of the box.

1.2 Common implementation ideas of plug-in

Taking java as an example, here are some commonly used plug-in implementation ideas combined with practical experience:

spi mechanism;
Agreed configuration and directory, using reflection to achieve;
Factories mechanism in springboot;
java agent (probe) technology;
spring built-in extension point;
Third-party plug-in package, for example: spring-plugin-core;
spring aop technology;
...

2. Java common plug-in implementation scheme

2.1 service loader method

serviceloader is the implementation of the spi mode provided by java. Develop and implement classes according to the interface, and then configure it. Java uses ServiceLoader to realize the sequential invocation of different implementations of the unified interface. The most classic use of serviceloader in java is the spi mechanism of Java.

2.1.1 java spi

The full name of SPI is Service Provider Interface, which is a built-in service discovery mechanism of JDK. SPI is a dynamic replacement and extension mechanism. For example, if there is an interface, if you want to dynamically add an implementation to it at runtime, you only need to add one to it according to the specification. Just implement the class. For example, the Driver interface in jdbc that everyone is familiar with, different manufacturers can provide different implementations, including mysql and oracle, and Java's SPI mechanism can find a service implementation for a certain interface.

The following is a simple diagram to illustrate the principle of the SPI mechanism

è¿éæå¥å¾çæè¿°

2.1.2 Simple case of java spi

In the following project directory, a plug-in interface is defined in an application project. In order to realize this interface, other application projects only need to import the jar package dependency of the current project for implementation. Here, for demonstration purposes, I will directly put different implementations in under the same project;

define interface

public interface MessagePlugin {

    public String sendMsg(Map msgMap);

}

define two different implementations

public class AliyunMsg implements MessagePlugin {

    @Override
    public String sendMsg(Map msgMap) {
        System.out.println("aliyun sendMsg");
        return "aliyun sendMsg";
    }
}

public class TencentMsg implements MessagePlugin {

    @Override
    public String sendMsg(Map msgMap) {
        System.out.println("tencent sendMsg");
        return "tencent sendMsg";
    }
}

Create a file directory in the resources directory according to the specification requirements, and fill in the full class name of the implementation class

Custom service loading class

 public static void main(String[] args) {
        ServiceLoader<MessagePlugin> serviceLoader = ServiceLoader.load(MessagePlugin.class);
        Iterator<MessagePlugin> iterator = serviceLoader.iterator();
        Map map = new HashMap();
        while (iterator.hasNext()){
            MessagePlugin messagePlugin = iterator.next();
            messagePlugin.sendMsg(map);
        }
    }

After running the above program, you can see the following effect. That is to say, the use of ServiceLoader can be loaded into the implementation of different interfaces. In the business, you only need to combine the configuration parameters according to your own needs to flexibly control the specific use. Which one achieves.

2.2 Custom configuration convention method

Serviceloader is actually flawed. In use, the interface name file must be defined in META-INF, and the class name of the implementation class can only be written in the file. If there are many plug-in things in a project, it is likely to appear The case of more and more profiles. Therefore, when using it in combination with actual projects, the following implementation ideas can be considered:

A application definition interface;
B, C, D and other application definition service implementation;
After B, C, and D applications are implemented, the SDK jar is achieved;
A application references the SDK or puts the SDK in a directory that can be read;
A application reads and parses the implementation class in the SDK;

On the basis of the above case, we make the following adjustments;

2.2.1 Add configuration file

In the configuration file, configure the specific implementation class

server :
  port : 8081
impl:
  name : com.congge.plugins.spi.MessagePlugin
  clazz :
    - com.congge.plugins.impl.TencentMsg
    - com.congge.plugins.impl.AliyunMsg

2.2.2 Custom configuration file loading class

Through this class, the implementation class in the above configuration file is encapsulated into a class object, which is convenient for subsequent use;

import lombok.Getter;
import lombok.Setter;
import lombok.ToString;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties("impl")
@ToString
public class ClassImpl {
    @Getter
    @Setter
    String name;

    @Getter
    @Setter
    String[] clazz;
}

2.2.3 Custom test interface

Use the above-mentioned encapsulated object to dynamically introduce it in the program through class loading

import com.congge.config.ClassImpl;
import com.congge.plugins.spi.MessagePlugin;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.HashMap;

@RestController
public class SendMsgController {

    @Autowired
    ClassImpl classImpl;

    //localhost:8081/sendMsg
    @GetMapping("/sendMsg")
    public String sendMsg() throws Exception{
        for (int i=0;i<classImpl.getClazz().length;i++) {
            Class pluginClass= Class.forName(classImpl.getClazz()[i]);
            MessagePlugin messagePlugin = (MessagePlugin) pluginClass.newInstance();
            messagePlugin.sendMsg(new HashMap());
        }
        return "success";
    }

}

2.2.4 Startup class

@EnableConfigurationProperties({ClassImpl.class})
@SpringBootApplication
public class PluginApp {

    public static void main(String[] args) {
        SpringApplication.run(PluginApp.class,args);
    }

}

After starting the project code, call the interface: localhost:8081/sendMsg , you can see the following output information in the console, that is, this method can also implement a method similar to serviceloader, but in actual use, it can be flexibly combined with configuration parameters control;

2.3 Customize the way to configure and read dependent jars

Furthermore, in many scenarios, we may not want to directly introduce the dependent package of the interface implementation into the project. At this time, we can consider reading the dependent jar in the specified directory and using reflection to dynamically load it. This is also a production One of the more commonly used practical experience.

In practice, the main steps are as follows:

Application A defines the service interface;
Application B, C, D, etc. implement the interface (or implement the same interface inside the application);
Application B, C, and D are packaged into jars and placed in the reading directory agreed by application A;
Application A loads the jar under the agreed directory, and loads the target method through reflection;

On the basis of the above, implement it according to the above implementation ideas;

2.3.1 Create a convention directory

Create a lib directory under the current project and put the dependent jars into it

2.3.2 Add a tool class for reading jar

Add a tool class to read the jar in the specified directory, and execute the reflection method by means of reflection, combined with the agreed configuration in the configuration file;

@Component
public class ServiceLoaderUtils {

    @Autowired
    ClassImpl classImpl;


    public static void loadJarsFromAppFolder() throws Exception {
        String path = "E:\\code-self\\bitzpp\\lib";
        File f = new File(path);
        if (f.isDirectory()) {
            for (File subf : f.listFiles()) {
                if (subf.isFile()) {
                    loadJarFile(subf);
                }
            }
        } else {
            loadJarFile(f);
        }
    }

    public static void loadJarFile(File path) throws Exception {
        URL url = path.toURI().toURL();
        // 可以获取到AppClassLoader，可以提到前面，不用每次都获取一次
        URLClassLoader classLoader = (URLClassLoader) ClassLoader.getSystemClassLoader();
        // 加载
        //Method method = URLClassLoader.class.getDeclaredMethod("sendMsg", Map.class);
        Method method = URLClassLoader.class.getMethod("sendMsg", Map.class);

        method.setAccessible(true);
        method.invoke(classLoader, url);
    }

    public  void main(String[] args) throws Exception{
        System.out.println(invokeMethod("hello"));;
    }

    public String doExecuteMethod() throws Exception{
        String path = "E:\\code-self\\bitzpp\\lib";
        File f1 = new File(path);
        Object result = null;
        if (f1.isDirectory()) {
            for (File subf : f1.listFiles()) {
                //获取文件名称
                String name = subf.getName();
                String fullPath = path + "\\" + name;
                //执行反射相关的方法
                //ServiceLoaderUtils serviceLoaderUtils = new ServiceLoaderUtils();
                //result = serviceLoaderUtils.loadMethod(fullPath);
                File f = new File(fullPath);
                URL urlB = f.toURI().toURL();
                URLClassLoader classLoaderA = new URLClassLoader(new URL[]{urlB}, Thread.currentThread()
                        .getContextClassLoader());
                String[] clazz = classImpl.getClazz();
                for(String claName : clazz){
                    if(name.equals("biz-pt-1.0-SNAPSHOT.jar")){
                        if(!claName.equals("com.congge.spi.BitptImpl")){
                            continue;
                        }
                        Class<?> loadClass = classLoaderA.loadClass(claName);
                        if(Objects.isNull(loadClass)){
                            continue;
                        }
                        //获取实例
                        Object obj = loadClass.newInstance();
                        Map map = new HashMap();
                        //获取方法
                        Method method=loadClass.getDeclaredMethod("sendMsg",Map.class);
                        result = method.invoke(obj,map);
                        if(Objects.nonNull(result)){
                            break;
                        }
                    }else if(name.equals("miz-pt-1.0-SNAPSHOT.jar")){
                        if(!claName.equals("com.congge.spi.MizptImpl")){
                            continue;
                        }
                        Class<?> loadClass = classLoaderA.loadClass(claName);
                        if(Objects.isNull(loadClass)){
                            continue;
                        }
                        //获取实例
                        Object obj = loadClass.newInstance();
                        Map map = new HashMap();
                        //获取方法
                        Method method=loadClass.getDeclaredMethod("sendMsg",Map.class);
                        result = method.invoke(obj,map);
                        if(Objects.nonNull(result)){
                            break;
                        }
                    }
                }
                if(Objects.nonNull(result)){
                    break;
                }
            }
        }
        return result.toString();
    }

    public Object loadMethod(String fullPath) throws Exception{
        File f = new File(fullPath);
        URL urlB = f.toURI().toURL();
        URLClassLoader classLoaderA = new URLClassLoader(new URL[]{urlB}, Thread.currentThread()
                .getContextClassLoader());
        Object result = null;
        String[] clazz = classImpl.getClazz();
        for(String claName : clazz){
            Class<?> loadClass = classLoaderA.loadClass(claName);
            if(Objects.isNull(loadClass)){
                continue;
            }
            //获取实例
            Object obj = loadClass.newInstance();
            Map map = new HashMap();
            //获取方法
            Method method=loadClass.getDeclaredMethod("sendMsg",Map.class);
            result = method.invoke(obj,map);
            if(Objects.nonNull(result)){
                break;
            }
        }
        return result;
    }


    public static String invokeMethod(String text) throws Exception{
        String path = "E:\\code-self\\bitzpp\\lib\\miz-pt-1.0-SNAPSHOT.jar";
        File f = new File(path);
        URL urlB = f.toURI().toURL();
        URLClassLoader classLoaderA = new URLClassLoader(new URL[]{urlB}, Thread.currentThread()
                .getContextClassLoader());
        Class<?> product = classLoaderA.loadClass("com.congge.spi.MizptImpl");
        //获取实例
        Object obj = product.newInstance();
        Map map = new HashMap();
        //获取方法
        Method method=product.getDeclaredMethod("sendMsg",Map.class);
        //执行方法
        Object result1 = method.invoke(obj,map);
        // TODO According to the requirements , write the implementation code.
        return result1.toString();
    }

    public static String getApplicationFolder() {
        String path = ServiceLoaderUtils.class.getProtectionDomain().getCodeSource().getLocation().getPath();
        return new File(path).getParent();
    }



}

2.3.3 Add test interface

Add the following test interface

    @GetMapping("/sendMsgV2")
    public String index() throws Exception {
        String result = serviceLoaderUtils.doExecuteMethod();
        return result;
    }

After all the above is completed, start the project and test the interface, and you can still get the expected results;

The above-mentioned implementation is still relatively rough. In actual use, more optimization and improvements are needed to meet actual business needs. For example, the interface input type parameter is used to control which dependent package is used for execution, etc.;

3. Plug-in implementation in SpringBoot

In the springboot framework that everyone uses more, in fact, the framework itself provides a lot of extension points, among which the most suitable for plug-in extension is the implementation of spring.factories;

3.1 SPI mechanism in Spring Boot

There is also a loading mechanism similar to Java SPI in Spring. It configures the implementation class name of the interface in the META-INF/spring.factories file, and then reads and instantiates these configuration files in the program. This custom SPI mechanism is the basis for the Spring Boot Starter implementation.

3.2 Implementation Principle of Spring Factories

The SpringFactoriesLoader class is defined in the spring-core package, which implements the function of retrieving the META-INF/spring.factories file and obtaining the configuration of the specified interface. Two external methods are defined in this class:

loadFactories obtains an instance of its implementation class according to the interface class, and this method returns a list of objects;
loadFactoryNames obtains the name of its interface class according to the interface, and this method returns a list of class names;

The key to the above two methods is to obtain the spring.factories file from the specified ClassLoader, and parse to get the list of class names. The specific code is as follows:

public static List<String> loadFactoryNames(Class<?> factoryClass, ClassLoader classLoader) {
    String factoryClassName = factoryClass.getName();
    try {
        Enumeration<URL> urls = (classLoader != null ? classLoader.getResources(FACTORIES_RESOURCE_LOCATION) :
                ClassLoader.getSystemResources(FACTORIES_RESOURCE_LOCATION));
        List<String> result = new ArrayList<String>();
        while (urls.hasMoreElements()) {
            URL url = urls.nextElement();
            Properties properties = PropertiesLoaderUtils.loadProperties(new UrlResource(url));
            String factoryClassNames = properties.getProperty(factoryClassName);
            result.addAll(Arrays.asList(StringUtils.commaDelimitedListToStringArray(factoryClassNames)));
        }
        return result;
    }
    catch (IOException ex) {
        throw new IllegalArgumentException("Unable to load [" + factoryClass.getName() +
                "] factories from location [" + FACTORIES_RESOURCE_LOCATION + "]", ex);
    }
}

From the code, we can know that in this method, the spring.factories files under all jar packages in the entire ClassLoader will be traversed, that is to say, we can configure the spring.factories files in our own jar without affecting the configuration in other places. It will not be overwritten by other people's configuration.

spring.factories is obtained by parsing Properties, so the content in the file we write is configured in the following way:

com.xxx.interface=com.xxx.classname

If an interface wants to configure multiple implementation classes, you can use ',' to split

3.3 Spring Factories case implementation

Next, look at a specific case implementation to experience the use of Spring Factories;

3.3.1 Define a service interface

Customize an interface and add a method to it;

public interface SmsPlugin {

    public void sendMessage(String message);

}

3.3.2 Define 2 service implementations

Implement class 1

public class BizSmsImpl implements SmsPlugin {

    @Override
    public void sendMessage(String message) {
        System.out.println("this is BizSmsImpl sendMessage..." + message);
    }
}

Implement class 2

public class SystemSmsImpl implements SmsPlugin {

    @Override
    public void sendMessage(String message) {
        System.out.println("this is SystemSmsImpl sendMessage..." + message);
    }
}

3.3.3 Add spring.factories file

In the resources directory, create a directory named: META-INF, and then define a spring.factories configuration file in this directory, the content is as follows, in fact, it is configured with the service interface and the full class name of the two implementation classes path;

com.congge.plugin.spi.SmsPlugin=\
com.congge.plugin.impl.SystemSmsImpl,\
com.congge.plugin.impl.BizSmsImpl

3.3.4 Add custom interface

Add a custom interface, have you found that this is a bit similar to java's spi, except that it is replaced by SpringFactoriesLoader to load services;

    @GetMapping("/sendMsgV3")
    public String sendMsgV3(String msg) throws Exception{
        List<SmsPlugin> smsServices= SpringFactoriesLoader.loadFactories(SmsPlugin.class, null);
        for(SmsPlugin smsService : smsServices){
            smsService.sendMessage(msg);
        }
        return "success";
    }

After starting the project, call this interface to test, localhost:8087/sendMsgV3?msg=hello , through the console, you can see that this method can correctly obtain the service implementation available in the system;

Using this mechanism of spring, some business logic in the system can be well extended and realized through the plug-in interface;

4. Actual cases of plug-in mechanism

Combined with the theoretical knowledge mastered above, the following is a complete operation step based on the Java SPI mechanism that is close to the real usage scenario;

4.1 Case Background

There are 3 microservice modules, and there is a plug-in interface in module A;
In an interface in module A, it is necessary to call a plug-in service to send SMS;
You can specify the specific way to send SMS through the configuration parameters of the configuration file;
If it is not loaded into any plugin, it will use the default SMS implementation of module A;

4.1.1 Module structure

1. biz-pp, plug-in interface project;

2. bitpt, aliyun SMS sending;

3. miz-pt, tencent SMS sending;

4.1.2 Overall Implementation Ideas

The complete implementation idea of this case is as follows:

biz-pp defines the service interface, and provides the jar to be relied on by other implementation projects;
bitpt and miz-pt rely on the jar of biz-pp and implement the methods in SPI;
After bitpt and miz-pt are implemented according to the API specification, they are packaged into jar packages or installed in the warehouse;
biz-pp depends on the bitpt and miz-pt jars in the pom, or you can get a specific implementation by starting and loading;

4.2 biz-pp key code implementation process

4.2.1 Add service interface

public interface MessagePlugin {

    public String sendMsg(Map msgMap);

}

4.2.2 Make a jar package and install it in the warehouse

This step is relatively simple and will not be expanded.

4.2.3 Custom Service Loading Tool Class

This class can be understood as in the real business code, according to the rules of the business definition, which plug-in implementation class can be loaded to send SMS;

import com.congge.plugin.spi.MessagePlugin;
import com.congge.spi.BitptImpl;
import com.congge.spi.MizptImpl;

import java.util.*;

public class PluginFactory {

    public void installPlugin(){
        Map context = new LinkedHashMap();
        context.put("_userId","");
        context.put("_version","1.0");
        context.put("_type","sms");
        ServiceLoader<MessagePlugin> serviceLoader = ServiceLoader.load(MessagePlugin.class);
        Iterator<MessagePlugin> iterator = serviceLoader.iterator();
        while (iterator.hasNext()){
            MessagePlugin messagePlugin = iterator.next();
            messagePlugin.sendMsg(context);
        }
    }

    public static MessagePlugin getTargetPlugin(String type){
        ServiceLoader<MessagePlugin> serviceLoader = ServiceLoader.load(MessagePlugin.class);
        Iterator<MessagePlugin> iterator = serviceLoader.iterator();
        List<MessagePlugin> messagePlugins = new ArrayList<>();
        while (iterator.hasNext()){
            MessagePlugin messagePlugin = iterator.next();
            messagePlugins.add(messagePlugin);
        }
        MessagePlugin targetPlugin = null;
        for (MessagePlugin messagePlugin : messagePlugins) {
            boolean findTarget = false;
            switch (type) {
                case "aliyun":
                    if (messagePlugin instanceof BitptImpl){
                        targetPlugin = messagePlugin;
                        findTarget = true;
                        break;
                    }
                case "tencent":
                    if (messagePlugin instanceof MizptImpl){
                        targetPlugin = messagePlugin;
                        findTarget = true;
                        break;
                    }
            }
            if(findTarget) break;
        }
        return targetPlugin;
    }

    public static void main(String[] args) {
        new PluginFactory().installPlugin();
    }


}

4.2.4 Custom interface

@RestController
public class SmsController {

    @Autowired
    private SmsService smsService;

    @Autowired
    private ServiceLoaderUtils serviceLoaderUtils;

    //localhost:8087/sendMsg?msg=sendMsg
    @GetMapping("/sendMsg")
    public String sendMessage(String msg){
        return smsService.sendMsg(msg);
    }

}

4.2.5 Interface implementation

@Service
public class SmsService {

    @Value("${msg.type}")
    private String msgType;

    @Autowired
    private DefaultSmsService defaultSmsService;

    public String sendMsg(String msg) {
        MessagePlugin messagePlugin = PluginFactory.getTargetPlugin(msgType);
        Map paramMap = new HashMap();
        if(Objects.nonNull(messagePlugin)){
            return messagePlugin.sendMsg(paramMap);
        }
        return defaultSmsService.sendMsg(paramMap);
    }
}

4.2.6 Add service dependencies

In this module, it is necessary to introduce jar dependencies on the two concretely implemented projects (or through the command method of starting loading)

    <dependencies>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <!--依赖具体的实现-->
        <dependency>
            <groupId>com.congge</groupId>
            <artifactId>biz-pt</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>

        <dependency>
            <groupId>com.congge</groupId>
            <artifactId>miz-pt</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>

    </dependencies>

This is the end of biz-pp's core code implementation, and we will continue with specific tests later;

4.3 Implementation process of bizpt key codes

Next is the specific SPI implementation process in the plug-in mechanism. The implementation steps of the two modules are exactly the same. Select one of them to explain. The project directory structure is as follows:

4.3.1 Add dependency on biz-app jar

Rely on the jar produced by the biz-app project above

    <dependencies>
        <dependency>
            <groupId>com.congge</groupId>
            <artifactId>biz-app</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
    </dependencies>

4.3.2 Add the implementation of the MessagePlugin interface

public class BitptImpl implements MessagePlugin {

    @Override
    public String sendMsg(Map msgMap) {
        Object userId = msgMap.get("userId");
        Object type = msgMap.get("_type");
        //TODO 参数校验
        System.out.println(" ==== userId :" + userId + ",type :" + type);
        System.out.println("aliyun send message success");
        return "aliyun send message success";
    }
}

4.3.3 Add SPI configuration file

According to the previous method, create a file in the resources directory, note that the file name is the full name of the interface in SPI, and the file content is the full class name of the implementation class

com.congge.spi.BitptImpl

4.3.4 Install the jar into the warehouse

After completing the coding of the implementation class, install the jar into the warehouse through the maven command, and then import it into the biz-app in the previous step;

4.4 Effect Demonstration

Start the biz-app service, call the interface: localhost:8087/sendMsg?msg=sendMsg, you can see the following effect

Why does this effect occur? Because we have configured which method to use for sending SMS in the implementation class, and we can find the corresponding service implementation when loading the plug-in, so it provides a better extension point for the current business.

Five, written at the end of the text

Judging from the current trend, the idea of plug-in mechanism has spread in various programming languages, frameworks, middleware, open source tools and other fields. Therefore, mastering the implementation mechanism of plug-in is very important for current program implementation or architecture design. The significance of is worthy of in-depth study, this is the end of this article, thank you for watching!

[Microservices] SpringBoot plug-in development model detailed summary

I. Introduction