一 什么是Extension 机制
Dubbo的类加载机制是模仿jdk的spi加载机制;
Jdk的SPI扩展加载机制:约定是当服务的提供者每增加一个接口的实现类时,需要在jar包的META-INF/service/目录下同时创建一个以服务接口命名的具体实现类,该文件里面就是保存的实现该接口的具体实现类。而当外部程序装配这个模块的时候,就能通过该jar包META-INF/services/里的配置文件找到具体的实现类名,并装载实例化,完成模块的注入。 基于这样一个约定就能很好的找到服务接口的实现类,而不需要再代码里制定。jdk提供服务实现查找的一个工具类:java.util.ServiceLoader。
比如:
jdbc:在jdbc4以前,开发人员还需要基于Class.forName("xxx")的方式来装载驱动。jdbc4也基于spi的机制来发现驱动提供商了,可以通过META-INF/services/java.sql.Driver文件里指定实现类的方式来暴露驱动提供者如下图:
但是原始的jdk扩展点加载机制有些缺陷:
·JDK标准的SPI会一次性实例化扩展点所有实现,如果有扩展实现初始化很耗时,但如果没用上也加载,会很浪费资源。
·如果扩展点加载失败,连扩展点的名称都拿不到了。比如:JDK标准的ScriptEngine,通过getName();获取脚本类型的名称,但如果RubyScriptEngine因为所依赖的jruby.jar不存在,导致RubyScriptEngine类加载失败,这个失败原因被吃掉了,和ruby对应不起来,当用户执行ruby脚本时,会报不支持ruby,而不是真正失败的原因。
·Dubbo增加了对扩展点IoC和AOP的支持,一个扩展点可以直接setter注入其它扩展点。
理解Dubbo的SPI机制之前的几个概念:
·扩展点:Dubbo作为一个非常灵活的框架,并不会强制所有用户必须使用Dubbo框架里自带的某些架构,比如注册中心的话,dubbo提供了zookeeper和redis,而开发者可以根据自己的需要使用自定的注册中心,针对这种可灵活替换的技术我们就称之为扩展点技术,类似扩展点在Dubbo中有很多,比如Protocol,Filter,LoadBlance,Cluster等等;
·Wrapper:Dubbo在加载某个接口的扩展类时候,如果发现某个实现中有一个拷贝构造函数,那么该接口实现是就是该接口的包装类,此时dubbo会在真正的实现类上包装一层wrapper,比如ProtocolFilterWrapper中含有Protocol引用,及构造函数,所以加载filter配置时会返回wrapper类;即这个时候从ExtensionLoader中返回的实际扩展类是被Wrapper包装的接口实现类。
@SPI 注解 :可以认为是定义默认实现类;
比如 Protocol 接口中,定义默认协议时 dubbo;
@SPI("dubbo")
public interface Protocol {}
@Adaptive 注解:该注解打在接口方法上;调 ExtensionLoader.getAdaptiveExtension()获取设配类,会先通过前面的过程生成 java 的源代码,在通过编译器编译成 class 加载。但是 Compiler 的实现策略选择也是通过 通过编译器编译成 class 文件那岂不是要死循环下去了吗?
此时分析 ExtensionLoader.getAdaptiveExtension()函数,对于有实现类上去打了注解 @Adaptive的dubbo spi扩展机制,它获取设配类不在通过前面过程生成设配类java源代码, 而是在读取扩展文件的时候遇到实现类打了注解@Adaptive 就把这个类作为设配类缓存在 ExtensionLoader 中,调用是直接返回。
·扩展点自动激活:可以理解为条件激活,比如Filter接口有很多扩展点实现类,当想简化配置用到哪些过滤器时,可以@Activate自动激活,或者配置为@Activate(“xxx”)条件配置激活。
二 ExtensionLoader过程详解
在Dubbo的扩展点加载机制中,ExtensionLoader是整个SPI加载的核心,而在Dubbo中ExtensionLoader的调用一般如下:
private static final ExtensionLoader<Container> loader = ExtensionLoader.getExtensionLoader(Container.class);
Container:
@SPI("spring")
public interface Container {
/**
* start.
*/
void start();
/**
* stop.
*/
void stop();
}
基于以下启动类的流程进行ExtensionLoader过程讲解(这种加载机制dubbo用的很多,后面的proctol也用到,为了条理清晰点,从启动开始梳理):
public class DemoProvider {
public static void main(String[] args) {
com.alibaba.dubbo.container.Main.main(args);
}
}
main启动类:
/*
* Copyright 1999-2011 Alibaba Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.dubbo.container;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Date;
import java.util.List;
import com.alibaba.dubbo.common.Constants;
import com.alibaba.dubbo.common.extension.ExtensionLoader;
import com.alibaba.dubbo.common.logger.Logger;
import com.alibaba.dubbo.common.logger.LoggerFactory;
import com.alibaba.dubbo.common.utils.ConfigUtils;
/**
* Main. (API, Static, ThreadSafe)
*
* @author william.liangf
*/
public class Main {
public static final String CONTAINER_KEY = "dubbo.container";
public static final String SHUTDOWN_HOOK_KEY = "dubbo.shutdown.hook";
private static final Logger logger = LoggerFactory.getLogger(Main.class);
//获取容器类的扩展点加载器
private static final ExtensionLoader<Container> loader = ExtensionLoader.getExtensionLoader(Container.class);
private static volatile boolean running = true;
public static void main(String[] args) {
try {
if (args == null || args.length == 0) {
String config = ConfigUtils.getProperty(CONTAINER_KEY, loader.getDefaultExtensionName());
args = Constants.COMMA_SPLIT_PATTERN.split(config);
}
final List<Container> containers = new ArrayList<Container>();
for (int i = 0; i < args.length; i ++) {
containers.add(loader.getExtension(args[i]));
}
logger.info("Use container type(" + Arrays.toString(args) + ") to run dubbo serivce.");
if ("true".equals(System.getProperty(SHUTDOWN_HOOK_KEY))) {
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
for (Container container : containers) {
try {
container.stop();
logger.info("Dubbo " + container.getClass().getSimpleName() + " stopped!");
} catch (Throwable t) {
logger.error(t.getMessage(), t);
}
synchronized (Main.class) {
running = false;
Main.class.notify();
}
}
}
});
}
for (Container container : containers) {
container.start();
logger.info("Dubbo " + container.getClass().getSimpleName() + " started!");
}
System.out.println(new SimpleDateFormat("[yyyy-MM-dd HH:mm:ss]").format(new Date()) + " Dubbo service server started!");
} catch (RuntimeException e) {
e.printStackTrace();
logger.error(e.getMessage(), e);
System.exit(1);
}
synchronized (Main.class) {
while (running) {
try {
Main.class.wait();
} catch (Throwable e) {
}
}
}
}
}
1. 获取扩展点加载器的方法
private static final ExtensionLoader<Container> loader = ExtensionLoader.getExtensionLoader(Container.class);
private static <T> boolean withExtensionAnnotation(Class<T> type) {
return type.isAnnotationPresent(SPI.class);
}
@SuppressWarnings("unchecked")
public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) {
//1.判断该class是否为空
if (type == null)
throw new IllegalArgumentException("Extension type == null");
//2.判断该class是否为接口类型
if(!type.isInterface()) {
throw new IllegalArgumentException("Extension type(" + type + ") is not interface!");
}
//3.判断该class是否是@SPI注解类型
if(!withExtensionAnnotation(type)) {
throw new IllegalArgumentException("Extension type(" + type +
") is not extension, because WITHOUT @" + SPI.class.getSimpleName() + " Annotation!");
}
//创建一个class对应的扩展点加载程序
ExtensionLoader<T> loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
//如果该loader为null,则创建一个class对应的extensionloader加载器放入EXTENSION_LOADERS中,
// 并且返回loader
if (loader == null) {
EXTENSION_LOADERS.putIfAbsent(type, new ExtensionLoader<T>(type));
loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
}
return loader;
}
private ExtensionLoader(Class<?> type) {
this.type = type;
// EXTENSION_LOADERS 静态的线程安全map,每个class类对应一个extensionloader加载器
objectFactory = (type == ExtensionFactory.class ? null : ExtensionLoader.getExtensionLoader(ExtensionFactory.class).getAdaptiveExtension());
}
getAdaptiveExtension()::
@SuppressWarnings("unchecked")
public T getAdaptiveExtension() {
//获取一个自适应缓存对象
Object instance = cachedAdaptiveInstance.get();
if (instance == null) {
if(createAdaptiveInstanceError == null) {
synchronized (cachedAdaptiveInstance) {
instance = cachedAdaptiveInstance.get();
if (instance == null) {
try {
//没有就自行创建一个AdaptiveExtension对象
instance = createAdaptiveExtension();
cachedAdaptiveInstance.set(instance);
} catch (Throwable t) {
createAdaptiveInstanceError = t;
throw new IllegalStateException("fail to create adaptive instance: " + t.toString(), t);
}
}
}
}
else {
throw new IllegalStateException("fail to create adaptive instance: " + createAdaptiveInstanceError.toString(), createAdaptiveInstanceError);
}
}
return (T) instance;
}
createAdaptiveExtension():
@SuppressWarnings("unchecked")
private T createAdaptiveExtension() {
try {
return injectExtension((T) getAdaptiveExtensionClass().newInstance());
} catch (Exception e) {
throw new IllegalStateException("Can not create adaptive extenstion " + type + ", cause: " + e.getMessage(), e);
}
}
private Class<?> getAdaptiveExtensionClass() {
getExtensionClasses();
if (cachedAdaptiveClass != null) {
return cachedAdaptiveClass;
}
return cachedAdaptiveClass = createAdaptiveExtensionClass();
}
getExtensionClasses()
private Map<String, Class<?>> getExtensionClasses() {
Map<String, Class<?>> classes = cachedClasses.get();
if (classes == null) {
synchronized (cachedClasses) {
classes = cachedClasses.get();
if (classes == null) {
classes = loadExtensionClasses();
cachedClasses.set(classes);
}
}
}
return classes;
}
injectExtension():
//为生成的 instance 注入变量;
//其目标是搜索所有 set 开头,同时只有一个入参的函数,执行该函数,对变量进行注入
private T injectExtension(T instance) {
try {
if (objectFactory != null) {
for (Method method : instance.getClass().getMethods()) {
if (method.getName().startsWith("set")
&& method.getParameterTypes().length == 1
&& Modifier.isPublic(method.getModifiers())) {
Class<?> pt = method.getParameterTypes()[0];
try {
String property = method.getName().length() > 3 ? method.getName().substring(3, 4).toLowerCase() + method.getName().substring(4) : "";
Object object = objectFactory.getExtension(pt, property);
if (object != null) {
method.invoke(instance, object);
}
} catch (Exception e) {
logger.error("fail to inject via method " + method.getName()
+ " of interface " + type.getName() + ": " + e.getMessage(), e);
}
}
}
}
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
return instance;
}
2.加载指定文件里的key-value值 例如:spring=com.alibaba.dubbo.container.spring.SpringContainer
loadExtensionClasses()
// 此方法已经getExtensionClasses方法同步过。
private Map<String, Class<?>> loadExtensionClasses() {
//1.获取注解为@SPI的类
final SPI defaultAnnotation = type.getAnnotation(SPI.class);
if(defaultAnnotation != null) {
//2.获取@SPI("")中的值
String value = defaultAnnotation.value();
if(value != null && (value = value.trim()).length() > 0) {
String[] names = NAME_SEPARATOR.split(value);
if(names.length > 1) {
throw new IllegalStateException("more than 1 default extension name on extension " + type.getName()
+ ": " + Arrays.toString(names));
}
if(names.length == 1) cachedDefaultName = names[0];
}
}
Map<String, Class<?>> extensionClasses = new HashMap<String, Class<?>>();
//加载META-INF/dubbo/internal/下的文件内容
loadFile(extensionClasses, DUBBO_INTERNAL_DIRECTORY);
//加载META-INF/dubbo/下的文件内容
loadFile(extensionClasses, DUBBO_DIRECTORY);
//加载META-INF/services/下的文件内容
loadFile(extensionClasses, SERVICES_DIRECTORY);
return extensionClasses;
}
loadFile():
private void loadFile(Map<String, Class<?>> extensionClasses, String dir) {
String fileName = dir + type.getName();
try {
//Enumeration(枚举)接口的作用和Iterator类似,
//只提供了遍历Vector和HashTable类型集合元素的功能,不支持元素的移除操作
Enumeration<java.net.URL> urls;
//创建一个ExtensionLoader加载器
ClassLoader classLoader = findClassLoader();
if (classLoader != null) {
urls = classLoader.getResources(fileName);
} else {
urls = ClassLoader.getSystemResources(fileName);
}
if (urls != null) {
//遍历Enumeration
while (urls.hasMoreElements()) {
java.net.URL url = urls.nextElement();
try {
//读取url文件中的内容
BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream(), "utf-8"));
try {
String line = null;
while ((line = reader.readLine()) != null) {
final int ci = line.indexOf('#');
if (ci >= 0) line = line.substring(0, ci);
line = line.trim();
if (line.length() > 0) {
try {
String name = null;
int i = line.indexOf('=');
if (i > 0) {
name = line.substring(0, i).trim();
line = line.substring(i + 1).trim();
}
if (line.length() > 0) {
Class<?> clazz = Class.forName(line, true, classLoader);
//判定此 Class 对象所表示的类或接口与指定的 Class 参数所表示的类或接口是否相同,
//或是否是其超类或超接口。如果是则返回 true;否则返回 false。
//如果该 Class 表示一个基本类型,且指定的 Class 参数正是该 Class 对象,则该方法返回 true;否则返回 false。
if (! type.isAssignableFrom(clazz)) {
throw new IllegalStateException("Error when load extension class(interface: " +
type + ", class line: " + clazz.getName() + "), class "
+ clazz.getName() + "is not subtype of interface.");
}
//isAnnotationPresent判断是否有@Adative
if (clazz.isAnnotationPresent(Adaptive.class)) {
if(cachedAdaptiveClass == null) {
cachedAdaptiveClass = clazz;
} else if (! cachedAdaptiveClass.equals(clazz)) {
throw new IllegalStateException("More than 1 adaptive class found: "
+ cachedAdaptiveClass.getClass().getName()
+ ", " + clazz.getClass().getName());
}
} else {
try {
clazz.getConstructor(type);
Set<Class<?>> wrappers = cachedWrapperClasses;
if (wrappers == null) {
cachedWrapperClasses = new ConcurrentHashSet<Class<?>>();
wrappers = cachedWrapperClasses;
}
wrappers.add(clazz);
} catch (NoSuchMethodException e) {
//获得构造器(Constructor)对象并调用其newInstance()方法创建对象
clazz.getConstructor();
if (name == null || name.length() == 0) {
name = findAnnotationName(clazz);
if (name == null || name.length() == 0) {
if (clazz.getSimpleName().length() > type.getSimpleName().length()
&& clazz.getSimpleName().endsWith(type.getSimpleName())) {
name = clazz.getSimpleName().substring(0, clazz.getSimpleName().length() - type.getSimpleName().length()).toLowerCase();
} else {
throw new IllegalStateException("No such extension name for the class " + clazz.getName() + " in the config " + url);
}
}
}
String[] names = NAME_SEPARATOR.split(name);
if (names != null && names.length > 0) {
//@Activate它有两个设置过滤条件的字段,group,value 都是字符数组。用来指定这个扩展类在什么条件下激活
Activate activate = clazz.getAnnotation(Activate.class);
if (activate != null) {
cachedActivates.put(names[0], activate);
}
for (String n : names) {
if (! cachedNames.containsKey(clazz)) {
cachedNames.put(clazz, n);
}
Class<?> c = extensionClasses.get(n);
if (c == null) {
extensionClasses.put(n, clazz);
} else if (c != clazz) {
throw new IllegalStateException("Duplicate extension " + type.getName() + " name " + n + " on " + c.getName() + " and " + clazz.getName());
}
}
}
}
}
}
} catch (Throwable t) {
IllegalStateException e = new IllegalStateException("Failed to load extension class(interface: " + type + ", class line: " + line + ") in " + url + ", cause: " + t.getMessage(), t);
exceptions.put(line, e);
}
}
} // end of while read lines
} finally {
reader.close();
}
} catch (Throwable t) {
logger.error("Exception when load extension class(interface: " +
type + ", class file: " + url + ") in " + url, t);
}
} // end of while urls
}
} catch (Throwable t) {
logger.error("Exception when load extension class(interface: " +
type + ", description file: " + fileName + ").", t);
}
}
extensionClasses的结果为:
{spring=class com.alibaba.dubbo.container.spring.SpringContainer, jetty=class com.alibaba.dubbo.container.jetty.JettyContainer, log4j=class com.alibaba.dubbo.container.log4j.Log4jContainer}
3.获取dubbo.properties的属性值
if (args == null || args.length == 0) {
//获取加载器中指定为dubbo.container的值
String config = ConfigUtils.getProperty(CONTAINER_KEY, loader.getDefaultExtensionName());
//将参数进行格式处理
args = Constants.COMMA_SPLIT_PATTERN.split(config);
}
@SuppressWarnings({ "unchecked", "rawtypes" })
public static String getProperty(String key, String defaultValue) {
//先从系统参数中寻找是否有dubbo.container的key
String value = System.getProperty(key);
if (value != null && value.length() > 0) {
return value;
}
//否则从自定义的properties中获取
Properties properties = getProperties();
return replaceProperty(properties.getProperty(key, defaultValue), (Map)properties);
}
getProperties():
public static Properties getProperties() {
if (PROPERTIES == null) {
synchronized (ConfigUtils.class) {
if (PROPERTIES == null) {
//默认在项目路径下去寻找dubbo.properties.file的进程变量值
String path = System.getProperty(Constants.DUBBO_PROPERTIES_KEY);
if (path == null || path.length() == 0) {
//寻找dubbo.properties.file的环境变量值
path = System.getenv(Constants.DUBBO_PROPERTIES_KEY);
if (path == null || path.length() == 0) {
//最后获取默认的dubbo.properties里的路径
path = Constants.DEFAULT_DUBBO_PROPERTIES;
}
}
//加载该路径下的属性值
PROPERTIES = ConfigUtils.loadProperties(path, false, true);
}
}
}
return PROPERTIES;
}
loadProperties():
public static Properties loadProperties(String fileName, boolean allowMultiFile, boolean optional) {
Properties properties = new Properties();
if (fileName.startsWith("/")) {
try {
FileInputStream input = new FileInputStream(fileName);
try {
properties.load(input);
} finally {
input.close();
}
} catch (Throwable e) {
logger.warn("Failed to load " + fileName + " file from " + fileName + "(ingore this file): " + e.getMessage(), e);
}
return properties;
}
List<java.net.URL> list = new ArrayList<java.net.URL>();
try {
Enumeration<java.net.URL> urls = ClassHelper.getClassLoader().getResources(fileName);
list = new ArrayList<java.net.URL>();
while (urls.hasMoreElements()) {
list.add(urls.nextElement());
}
} catch (Throwable t) {
logger.warn("Fail to load " + fileName + " file: " + t.getMessage(), t);
}
if(list.size() == 0) {
if (! optional) {
logger.warn("No " + fileName + " found on the class path.");
}
return properties;
}
if(! allowMultiFile) {
if (list.size() > 1) {
String errMsg = String.format("only 1 %s file is expected, but %d dubbo.properties files found on class path: %s",
fileName, list.size(), list.toString());
logger.warn(errMsg);
// throw new IllegalStateException(errMsg); // see http://code.alibabatech.com/jira/browse/DUBBO-133
}
// fall back to use method getResourceAsStream
try {
properties.load(ClassHelper.getClassLoader().getResourceAsStream(fileName));
} catch (Throwable e) {
logger.warn("Failed to load " + fileName + " file from " + fileName + "(ingore this file): " + e.getMessage(), e);
}
return properties;
}
logger.info("load " + fileName + " properties file from " + list);
for(java.net.URL url : list) {
try {
Properties p = new Properties();
InputStream input = url.openStream();
if (input != null) {
try {
p.load(input);
properties.putAll(p);
} finally {
try {
input.close();
} catch (Throwable t) {}
}
}
} catch (Throwable e) {
logger.warn("Fail to load " + fileName + " file from " + url + "(ingore this file): " + e.getMessage(), e);
}
}
return properties;
}
此时properties的结果为:
{dubbo.container=log4j,spring,
dubbo.service.loadbalance=roundrobin,
dubbo.registry.address=zookeeper://127.0.0.1:2181,
dubbo.application.owner=william,
dubbo.application.name=demo-provider,
dubbo.protocol.port=20880,
dubbo.protocol.name=dubbo}
4.添加dubbo.properties中指定的dubbo.container
final List<Container> containers = new ArrayList<Container>();
for (int i = 0; i < args.length; i ++) {
containers.add(loader.getExtension(args[i]));
}
private T createExtension(String name) {
Class<?> clazz = getExtensionClasses().get(name);
if (clazz == null) {
throw findException(name);
}
try {
T instance = (T) EXTENSION_INSTANCES.get(clazz);
if (instance == null) {
EXTENSION_INSTANCES.putIfAbsent(clazz, (T) clazz.newInstance());
instance = (T) EXTENSION_INSTANCES.get(clazz);
}
injectExtension(instance);
Set<Class<?>> wrapperClasses = cachedWrapperClasses;
if (wrapperClasses != null && wrapperClasses.size() > 0) {
for (Class<?> wrapperClass : wrapperClasses) {
instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance));
}
}
return instance;
} catch (Throwable t) {
throw new IllegalStateException("Extension instance(name: " + name + ", class: " +
type + ") could not be instantiated: " + t.getMessage(), t);
}
}
Class<?> clazz = getExtensionClasses().get(name);获取对应key的class,
{spring=class com.alibaba.dubbo.container.spring.SpringContainer, jetty=class com.alibaba.dubbo.container.jetty.JettyContainer, log4j=class com.alibaba.dubbo.container.log4j.Log4jContainer}
测试:如果我在后面添加一个logback的容器,dubbo.container=log4j,spring, logback,此时运行报错
原因找不到该为logback的容器。
小结:
由于从消费者启动容器开始梳理源码,有些细节性的还没涉及到,比如 扩展点的 Wrapper 类,@Adaptive的注解在dubbo中具体的作用,在后续的源码解析流程中会细说。