dubbo core source code - SPI and IOC, AOP

Dubbo core source code analysis

Dubbo has many designs worth learning and learning from. A few points to understand:

• SPI mechanism
• Adaptive extension point
• Ioc and Aop
• How Dubbo integrates with Spring

SPI of Dubbo core

​ In the source code of Dubbo, there are the following three types of codes in many places, which are adaptive extension points, extension points with specified names, and activation extension points:

ExtensionLoader.getExtensionLoader(XXX.class).getAdaptiveExtension();
ExtensionLoader.getExtensionLoader(XXX.class).getExtension(name);
ExtensionLoader.getExtensionLoader(XXX.class).getActivateExtension(url,key);

​ This extension point is actually the SPI mechanism in Dubbo. Regarding SPI, I don’t know if you still have an impression. The SpringFactoiesLoader automatically assembled by Springboot is also an SPI mechanism.

java SPI extension point implementation

The full name of SPI is Service Provider Interface. It was originally a service discovery mechanism built into JDK. It is mainly used for the extension and implementation of services. The SPI mechanism is used in many scenarios, such as database connection. JDK provides java.sql.Driver interface. This driver class is not implemented in JDK, but is implemented by different database vendors, such as Oracle and Mysql. The driver package will implement this interface, and then JDK uses the SPI mechanism to find the corresponding driver from the classpath to obtain the connection to the specified database. This plug-in extension loading method also follows certain protocol agreements. For example, all extension points must be placed in the resources/META-INF/services directory, and the SPI mechanism will scan the property files in this path by default to complete load.

Here is a chestnut:

• Create a common Maven project Driver and define an interface. This interface is only a specification and has not been implemented. The implementation is provided by third-party manufacturers.

public interface Driver {
    
    
    String connect();
}

• Create another common Maven project Mysql-Driver, add Driver's Maven dependency

<dependency>
    <groupId>com.scmpe.book.spi</groupId>
    <artifactId>Driver</artifactId>
    <version>1.0-SNAPSHOT</version>
</dependency>

• Create MysqlDriver and implement the Driver interface, which represents a third-party extension implementation

public class MysqlDriver implements Driver {
    
    
    @Override
    public String connect() {
    
    
        return "连接Mysql 数据库";
    }
}

• Create a file com.scmpe.book.spi.Driver named after the full path of the Driver interface in the resources/META-INF/services directory, and fill in the Driver implementation class in it

  com.scmpe.book.spi.MysqlDriver
  

• Create a test class and load it with ServiceLoader

@Test
public void connectTest() {
    
    
    ExtensionLoader<Driver> extensionLoader = ExtensionLoader.getExtensionLoder(Driver.class);
    Driver driver = extensionLoader.getExtension("mysqlDriver");
    System.out.println(driver.connect());
}

Dubbo SPI extension point source code

​ Earlier we used to ExtensionLoader.getExtensionLoader.getExtension() demonstrate the usage of SPI in Dubbo. Next, we will analyze how to implement SPI in Dubbo source code based on this method.

​ This code is divided into two parts: first, we ExtensionLoader.getExtensionLoader get an ExtensionLoader instance through , and then getExtension()get the extension point with the specified name through the method. Let's analyze the first part first.

ExtensionLoder.getExtensionLoader

This method is used to return an ExtensionLoader instance, the main logic is:

1. First obtain the ExtensionLoader corresponding to the extension class from the cache;
2. If the cache misses, create a new instance and save it to the EXTENSION_LOADERS collection for cache.
3. In the ExtensionLoader construction method, an objectFactory is initialized, which will be used later, so let’s ignore it for now

public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) {
    
    
    // 省略部分代码
    ExtensionLoder<T> loder = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
    if (loder == null) {
    
    
        EXTENSION_LOADERS.putIfAbsent(type,new ExtensionLoader<T>(type));
        loder = (ExtensinoLoder<T>) EXTENSION_LOADERS.get(type);
    }
    return loader;
}
//构造方法
private ExtensionLoader(Class<?> type) {
    
    
    this.type = type;
    ObjectFactory = (type == ExtensionFactory.class ? null : ExtensionLoader.getExtensionLoader(ExtensionFactory.class).getAdaptiveExtension());
}

getExtension()

​ This method is used to obtain the corresponding extension point according to the specified name and return it. In the previous demonstration case, if the name is mysqlDriver, then the returned implementation class should be MysqlDriver.

• name is used to judge the parameters, and if name="true", a default extension implementation will be returned.

• Create a Holder object, the user caches the instance of the extension point.

• Create an extension point via createExtension(name) if it does not exist in the cache.

public T getExtension(String name) {
    
    
    if(StringUtils.isEpty(name)) {
    
    
        throw new IllegalArgumentException("Extension name = null");
    }
    if ("true".equals(name)) {
    
    
        return getDefaultExtension();
    }
    // 创建或者返回一个Holder对象，用于缓存实例
    final Holder<Object> holder = getOrCreateHolder(name);
    Object instance = holder.get();
    if (instance == null) {
    
     //如果缓存不存在，则创建一个实例
 		synchronized (holder) {
    
    
            instance = holder.get();
			if (instance == null) {
    
    
                instance = createExtension(name);
                holder.set(instance);
            }            
        }       
       
    }
    return (T)instance;
}

The meaning of the above code is to check the cache first, and if the cache misses, an extension object is created. It is not difficult to guess that createExtension() should be to find the implementation of the extension point corresponding to the name under the specified path, and return after instantiation.

• Get an extension class through getExtensionClasses().get(name)
• After being instantiated by reflection, it is cached in the EXTENSION_INSTANCES collection.
• injectExtension implements dependency injection
• Wrap the extended class object through Wrapper.

private T createExtension(String name) {
    
    
    Class<?> clazz = getExtensionClasses().get(name);
    if (clzz == null) {
    
    
        throw findException(name);
    }
    try {
    
    
        T instance = (T) EXTENSION_INSTANCES.get(clazz);
        if (instance == null) {
    
    
            EXTENSION_INSTANCES.putIfAbsent(clazz,clazz.newInstance());
            instance = (T) EXTENSION_INSTANCES.get(clazz);
        }
        //依赖注入
        injectExtension(instance);
        //通过Wrapper包装
        Set<Class<?>> WrapperClasses = cachedWrapperClasses;
        if (CollectUtils.isNotEmpty(wrapperClasses)) {
    
    
        	for(Class<?> wrapperClass : wrapperClasses) {
    
    
                instance = injectExtension((T) WrapperClass.getConstructor(type).newInstance(instance))
            }
        }
        initExtension(instance);
        return instance;
    } catch(Throwable t) {
    
    
        throw new IllegalStateException()
    }
}

• Obtain the loaded extension class from the cache
• If the cache is not hit, call loadExtensionClasses to load the extension class

private Map<String,Class<?>> getExtensionClasses() {
    
    
    Map<String,Class<?>> classes = cachedClasses.get();
    if (classes == null) {
    
    
        synchronized (cachedClasses) {
    
    
            classes = cachedClasses.get();
            if (classes == null) {
    
    
                classes = loadExtensionClasses();
                cachedClasses.set(classes);
            }
        }
    }
    return classes;
}

The code implementation routines in Dubbo are basically the same. First access the cache, and then load 2 extension classes through loadExtensionClasses after a cache miss. This method mainly does two things.

• Return the default extension object of the current extension interface through the cacheDefaultExtensionName method, and cache
• The configuration file in the specified file directory is loaded through the loadDirectory method.

private Map<String,Class<?>> loadExtensionclasses() {
    
    
	cacheDefaultExtensionName();//获得当前type接口默认的扩展类
	Map<String,Class<?>> extensionclasses = new HashMap<>();
    //解析指定路径下的文件
    loadDirectory(extensionclasses, DUBBO_INTERNAL_DIRECTORY, type.getName(), true);
    loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName().replace("org.apache", "com.alibaba"),true);
    loadDirectory(extensionclasses,DUBBO_DIRECTORY,type.getName());
    loadDirectory(extensionClasses, DUBBO_DIRECTORY,type.getName().replace("org.apache","com.alibaba"));
    loadDirectory(extensionclasses,SERVICES_DIRECTORY,type.getName());
    loadDirectory(extensionclasses,SERVICES_DIRECTORY,type.getName().replace("org.apache","com.alibaba"));
    return extensionClasses;
}

The logic of the loadDirectory method is relatively simple. It is to find the corresponding file from the specified directory according to the full path name of the type passed in, and load and save it into the extensionClasses collection after parsing the content.

The cacheDefaultExtensionName method is also relatively simple, but it has a certain relationship with the business.

• Get the @SPI annotation of the specified extension interface
• Get the name in the @SPI annotation and save it in the cachedDefaultName attribute.

private void cacheDefaultExtensionName() {
    
    
    final SPI defaultAnnotation = type.getAnnotation(SPI.class);
    if (defaultAnnotation == null) {
    
    
        return;
    }
    // 得到注解中的value值
    String value = defaultAnnotation.value();
    if (value = value.tirm().lenth > 0) {
    
    
        String[] names = NAME_SEPARATOR.spilt(value);
        if (names.length > 1) {
    
    
            throw new IllegalStateException()
        }
        if (names.lenth == 1) {
    
    
            cachedDefault = names[0];
        }
    }
}

Taking org.apache.dubbo.rpc.Protocol in Dubbo as an example, there is a default value of dubbo in the @SPI annotation, which means that if the protocol type is not explicitly specified, the Dubbo protocol is used by default to publish services.

@SPI("dubbo")
public interface Protocol {
    
    
    //...
}

This is the process of loading the extension class with the specified name in dubbo.

Adaptive extension point

​ An adaptive extension point can be understood as an adapter extension point. Simply put, it is possible to dynamically configure an extension class according to the context.

ExtensionLoder.getExtensionLoader(class).getAdaptiveExtension();

Adaptive extension points are declared through the @Adaptive annotation, which can be used in two ways

• The Adaptive annotation is defined on the class, indicating that the current class is an adaptive extension class

  @Adaptive
  public class AdativeCompiler implents Compiler {
        
        
      // 省略
  }
  

  The AdaptiveCompiler class is an adaptive extension class, and an instance of the AdaptiveCompiler class can be returned by ExtensionLoader.getExtensionLoader(Compiler.class).getAdaptiveExtension();

• The @Adaptive annotation is defined at the method level, and a dynamic bytecode will be generated through a dynamic proxy for adaptive matching. ,.

public interface Protocol {
    
    
    int getDefaultPort();
    
    @Adaptive
    <T> Exporter<T> export(Invoker<T> invoker) throws RpcException;
    @Adaptive
    <T> Invoker<T> refer(Class<T> type,URL url) throws RpcException;
}

Two methods in the Protocol extension class declare the @Adaptive annotation, which means that this is an adaptive method. There are many places in the Dubbo source code to obtain an adaptive extension point through the following line of code

Protocol protocol = ExtensionLoader.getExtensionLoader(Protocol.class).getAdaptiveExtension();

Next, analyze the source code implementation of Protocol-based adaptive extension point method ExtensionLoader.getExtensinoLoader(Protocol.class).getAdaptiveExtensino().

Judging from the source code, the getAdaptiveExtension method is very simple and only does two things:

• Get an adaptive extension point instance from the cache.
• If there is a cache miss, an adaptive extension point is created via createAdaptiveExtension.

public T getAdaptiveExtension() {
    
    
    Object instance = this.cachedAdaptiveInstance.get();
    if (instance == null) {
    
    
        if (this.createAdaptiveInstanceError != null) {
    
    
            throw new IllegalStateException();
        }
        // 创建自适应扩展点实例，并放置到缓存中
        synchronized(this.cachedAdativeInstance) {
    
    
            instance = this.cachedAdaptiveInstance.get();
            if (instance == null) {
    
    
                try {
    
    
                    instance = this.createAdaptiveExtension();
                    this.cachedAdaptiveInstance.set(instance);
                }
                }catch(Throwable var5) {
    
    
                this.createAdaptiveInstanceError = var5;
                throw new IllegalstateException("Failed to create adaptive instance:+var5.toString(), var5);

            }
            
        }
    }
    return instance;
}

According to the previous analysis of adaptive extension points, we can basically guess the implementation mechanism of the createAdaptiveExtension method,

• getAdaptiveExtensionClasses Get an instance of an adaptive extension class
• injectExtension completes dependency injection

private T createAdaptiveExtension( {
    
    
    try {
    
    
    return this.injectExtension(this.getAdaptiveExtensionClass().newInstance());}catch (Exception var2){
    
    
    throw new IllegalStateException("can't create adaptive extension " + this.type +", cause:" +var2.getMessage(), var2);
    }
}

InjectExtension will be analyzed later, first look at getAdaptiveExtensinoClass.

• Load all extension points of the current Chunru type through the getExtensionClasses method, and cache them in a collection
• If cachedAdaptiveClass is empty, call createAdaptiveExtensionCalss to create

private Class<?> getAdaptiveExtensionClass() {
    
    
    this.getExtensionClasses();
    return this.cachedAdaptiveClass != null ? this.cachedAdaptiveClass : (this.cachedAdaptiveClass = this.createAdaptiveExtensionClass());
}

The getExtensionClasses method has been mentioned before, directly look at the createAdaptiveExtensionClass method, which involves the generation and loading of dynamic bytecodes.

• code is a dynamically spliced ​​class.
• Perform dynamic compilation through Compiler.

private Class<?> createAdaptiveExtensionClass() {
    
    
    String code = (new AdaptiveClassCodeGenerator(this.type,this.cachedDefaultName)).generate();
    ClassLoader classLoader = findClassLoader();
    Compiler compiler = (Compiler)getExtensionLoader(Compiler.class).getAdaptiveExtension();
    return compiler.compile(code,classLoader);
}

In the adaptive extension point loading based on the Protocol interface, the string of code splicing at this time is as follows (for the sake of beautiful layout, some useless codes are removed)).

public class Protocol$Adaptive implements Protocol {
    
    
    //省略部分代码
    public Exporter export(Invoker arg0) throws org.apache. dubbo.rpc.RpcException {
    
    
    if (arge == null) throw new IllegalArgumentException("Invoker argument == null");
    if (arg0.getUrl()== nul1)
    	throw new IllegalArgumentException("Invoker argument getUrl() - null");
    URL url = arg0.getUr1();
    String extName = (url.getProtocol() == null ?"dubbo":url.getProtoco1());\
    if (extName == null)
    	throw new IllegalstateException("Failed to get extension （Protocol) name from”+url.toString() +")use keys([protocol])");
    //根据名称获得指定扩展点
    Protocol extension = ExtensionLoader.getExtensionLoader(Protocol.class).getExtension(extName);
    return extension. export(arg0);
}

public Invoker refer(Class arg0，URL arg1) throws RpcException {
    
    
    if (arg1 ==null) throw new IllegalArgumentException("url == null");
    URL url = arg1;
   	String extName =(url.getProtocol()== null ? "dubbo":url.getProtocol());
    if (extName == null) throw new IllegalstateException("Failed to get extension (Protocol) name from ur1("+ url.toString() +") use keys([protocol])");
    Protocol extension = ExtensionLoader.getExtensionLoader(Protocol.class).getExtension(extName):
    return extension.refer(arg0,arg1);
}

Protocol$Adaptive is a dynamically generated adaptive extension class that can be used in the following ways:

Protocol protocol=ExtensionLoader.getExtensionLoader(Protocol.class).getAdaptiveExtension();
protocol.export( ...);

When protocol.export() is called, the export method in the Protocol$Adaptive class is actually called. And this method is nothing more than obtaining the corresponding extension class through getExtension according to the protocol name configured by the Dubbo service.

IOC and AOP in Dubbo

IOC and AOP are no strangers to us. They are the core functions of the Spring Framework. In fact, these two mechanisms are also used in Dubbo. The following analyzes the embodiment of these two mechanisms one by one from the source code level.

IOC

​A very important idea in IoC is that when the system is running, it dynamically provides other objects it needs to an object. This mechanism is realized through DI (Dependency Injection).

When analyzing the Dubbo SPI mechanism, there is a piece of code in the createExtension method as follows:

private T createExtension(String name) {
    
    
    try {
    
    
        T instance = (T) EXTENSION_INSTANCES.get(clazz);
        if (instance == null) {
    
    
            EXTENSION_INSTANCES.putIfAbsent(clazz, clazz.newInstance());
            instance = (T)EXTENSION_INSTANCES.get(clazz);
        }
		injectExtension(instance);
        //省略部分代码
        return instance
    }catch(Throwable t) {
    
    
        //省略部分代码
    }
}

injectExtension is the implementation of dependency injection, and the overall logic is relatively simple.

• Traverse all set methods in the loaded extension class
• Get the parameter type in the set method, if the parameter type is an object type, then get the attribute name in the set method.
• Use the adaptive extension point to load the extension class corresponding to the attribute name.
• Call the set method to complete the assignment.

private T injectExtension(T instance) {
    
    
    if (objectFactory == null) {
    
    
        return instance;
    }
   	try {
    
    
        for (Method method : instance.getClass().getMethods()) {
    
    
            if (!isSetter(method)) {
    
    
                continue;
            }
            if (method.getAnnotation(DisableInject.class) != null) {
    
    
                continue;
            }
            // 获得扩展类中方法的参数类型
            Class<?> pt = method.getParameterTypes()[0];
            // 如果不是对象类型,跳过
            if (ReflectUtils.isPrimitives(pt)) {
    
    
                continue;
            }
            try {
    
    
                // 获取方法对应的属性名称
                String property = getSetterProperty(method);
                // 根据class及name，使用自适应扩展点加载并且通过set方法进行赋值
                Object object = objectFactory.getExtension(pt,property);
                if (object != null) {
    
    
                    method.invoke(instance,object);
                }
            }catch(Exception e) {
    
    
               logger.error("Failed to inject via method " + method.getName()+"of interface " + type.getName() +":" +e.getMessage(),e);
        	} 
        }catch(Exception e) {
    
    
                logger.error(e.getMessage,e);
        }
        return instance;
    }
}

To sum up, the main function of injectExtension is that if there is an object that needs to be injected in the currently loaded extension class (the object must provide a setter method), then it will be loaded and assigned through the adaptive extension point.

Take org.apache.dubbo.registry.integration.RegistryProtocol as an example, there is a Protocol member object in it, and the setProtocol method is provided for it, then when the RegistryProtocol extension class is loaded, it will automatically inject the instance of the protocol member attribute .

public class RegistryProtocol implements Protocol {
    
    
    //省略部分代码
    private Protocol protocol;
	public void setProtocol(Protocol protocol) {
    
    
        this.protocol = protocol;
    }
    //省略部分代码
}

AOP

​ AOP stands for Aspect Oriented Programming, which means aspect-oriented programming, which is a kind of thinking or programming paradigm. Its main intention is to separate business logic from functional logic, and then weave it during runtime or during class loading. The advantage of this is that the complexity of the code can be reduced and the reusability can be improved.

​ In the Dubbo API mechanism, the AOP design idea is also reflected in the createExtension method in the ExtensionLoader class.

private T createExtension(String name){
    
    
    //..
    try {
    
    
        //...
        Set<class<?>> wrapperClasses = cachedwrapperClasses;if (collectionutils.isNotEmpty(wrapperclasses)){
    
    
        for (class<?> wrapperClass : wrapperClasses){
    
    
            instance = injectExtension((T) wrapperclass.getConstructor(type).newInstance(instance));
            }
        }
        initExtension( instance);
        return instance;
    }catch (Throwable t){
    
    
        //...
    }
}

This code was mentioned in the previous chapter, carefully analyze the following line of code

instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance));

Among them, dependency injection and AOP ideas are used respectively. The embodiment of AOP idea is to wrap the original extension class instance based on Wrapper decorator class.