The details and evolution of the three mechanisms of JDK SPI, Spring SPI, and Dubbo SPI
SPI mechanism
Java SPI（Service Provider Interface）It is a service discovery mechanism provided by JDK for dynamically loading and extending service providers in applications at runtime.
The essence of SPI is to configure the fully qualified name of the interface implementation class in the file, and the service loader reads the configuration file and loads the implementation class. In this way, the implementation class can be dynamically replaced for the interface at runtime. Because of this feature, we can easily provide extended functions for our programs through the SPI mechanism.
[Example] Define the interface in Java
java.sql.Driver, and then use it directly, regardless of the implementation class. The specific implementation class is loaded through the SPI mechanism. Drivers include JDBC, ODBC, etc. Which driver Jar package we import, there will bejava.sql.Driverfiles in the META-INF/services directory of the Jar package, which storesjava.sql.Driverthe fully qualified name of the implementation class that implements the interface in the current Jar package .
When the service provider provides an interface implementation, classpath下的META-INF/services/a file named after the service interface needs to be created in the directory, and the content in this file is the specific implementation class of the interface. When other programs need this service, you can find the META-INF/services/configuration file in the jar package (usually the jar package is used as a dependency ). Load instantiation and the service is ready to use.
The tool class for finding the implementation of the service in the JDK is: java.util.ServiceLoader.
Application of SPI mechanism
Load the driver in JDBC
1. JDBC interface definition
First of all, the interface is defined in java java.sql.Driver, and there is no specific implementation. The specific implementations are provided by different manufacturers.
2. mysql implementation
In the jar package of mysql mysql-connector-java-6.0.6.jar, you can find the directory. There will be a file META-INF/servicesnamed in the directory . The content of the file is the implementation of the interface defined in Java.java.sql.Drivercom.mysql.cj.jdbc.Driver
3. postgresql implementation
The same configuration file can also be found in the same jar postgresqlpackage . The content of the file is that this is the implementation of Java .postgresql-42.0.0.jarorg.postgresql.Driverpostgresqljava.sql.Driver
4. Source code implementation
The search for the driver is actually in DriverManager, DriverManagerwhich is implemented in Java and used to obtain the database connection. DriverManagerThere is a static code block in , as follows:
static {
loadInitialDrivers();
println("JDBC DriverManager initialized");
}
You can see that the instantiation driver is loaded, and then look at loadInitialDriversthe method:
private static void loadInitialDrivers() {
String drivers;
try {
drivers = AccessController.doPrivileged(new PrivilegedAction<String>() {
public String run() {
return System.getProperty("jdbc.drivers");
}
});
} catch (Exception ex) {
drivers = null;
}
AccessController.doPrivileged(new PrivilegedAction<Void>() {
public Void run() {
//使用SPI的ServiceLoader来加载接口的实现
ServiceLoader<Driver> loadedDrivers = ServiceLoader.load(Driver.class);
Iterator<Driver> driversIterator = loadedDrivers.iterator();
try{
while(driversIterator.hasNext()) {
driversIterator.next();
}
} catch(Throwable t) {
// Do nothing
}
return null;
}
});
println("DriverManager.initialize: jdbc.drivers = " + drivers);
if (drivers == null || drivers.equals("")) {
return;
}
String[] driversList = drivers.split(":");
println("number of Drivers:" + driversList.length);
for (String aDriver : driversList) {
try {
println("DriverManager.Initialize: loading " + aDriver);
Class.forName(aDriver, true,
ClassLoader.getSystemClassLoader());
} catch (Exception ex) {
println("DriverManager.Initialize: load failed: " + ex);
}
}
}
The main steps of the above code are:
1. Obtain the definition of the driver from the system variable.
2. Use SPI to obtain the implementation of the driver.
3. Traverse the specific implementations obtained by using SPI, and instantiate each implementation class.
4. Instantiate the specific implementation class according to the driver list obtained in the first step.
Mainly focus on steps 2 and 3. These two steps are the usage of SPI. First look at the second step, using SPI to obtain the driver implementation. The corresponding code is:
ServiceLoader<Driver> loadedDrivers = ServiceLoader.load(Driver.class);
There is no META-INF/servicessearch for configuration files in the directory, nor loading of specific implementation classes. What we do is to encapsulate our interface type and class loader, and initialize an iterator. Then look at the third step, traverse the specific implementation obtained by using SPI, and instantiate each implementation class. The corresponding code is as follows:
//获取迭代器
Iterator<Driver> driversIterator = loadedDrivers.iterator();
//遍历所有的驱动实现
while(driversIterator.hasNext()) {
driversIterator.next();
}
When traversing, first call driversIterator.hasNext()the method, here will search classpathall the files in the META-INF/servicesdirectory and jar package java.sql.Driver, and find the name of the implementation class in the file. At this time, the specific implementation class is not instantiated (the ServiceLoaderspecific source code implementation is in under).
Then call driversIterator.next();the method. At this time, each implementation class will be instantiated according to the driver name. The driver is now found and instantiated.
Spring SPI
During springbootthe autowiring process, META-INF/spring.factoriesthe file is eventually loaded, and the process of loading is SpringFactoriesLoaderdone by loading. Search all configuration files from CLASSPATHeach Jar package below META-INF/spring.factories, then parse propertiesthe files, and return after finding the configuration with the specified name. It should be noted that, in fact, it will not only ClassPathsearch under the path, but also scan all Jar packages under the path, but this file will only Classpathbe in the next jar package.
public static final String FACTORIES_RESOURCE_LOCATION = "META-INF/spring.factories";
// spring.factories文件的格式为：key=value1,value2,value3
// 从所有的jar包中找到META-INF/spring.factories文件
// 然后从文件中解析出key=factoryClass类名称的所有value值
public static List<String> loadFactoryNames(Class<?> factoryClass, ClassLoader classLoader) {
String factoryClassName = factoryClass.getName();
// 取得资源文件的URL
Enumeration<URL> urls = (classLoader != null ? classLoader.getResources(FACTORIES_RESOURCE_LOCATION) : ClassLoader.getSystemResources(FACTORIES_RESOURCE_LOCATION));
List<String> result = new ArrayList<String>();
// 遍历所有的URL
while (urls.hasMoreElements()) {
URL url = urls.nextElement();
// 根据资源文件URL解析properties文件，得到对应的一组@Configuration类
Properties properties = PropertiesLoaderUtils.loadProperties(new UrlResource(url));
String factoryClassNames = properties.getProperty(factoryClassName);
// 组装数据，并返回
result.addAll(Arrays.asList(StringUtils.commaDelimitedListToStringArray(factoryClassNames)));
}
return result;
}
Below is the configuration Spring Bootin a paragraphspring.factories
# Logging Systems
org.springframework.boot.logging.LoggingSystemFactory=\
org.springframework.boot.logging.logback.LogbackLoggingSystem.Factory,\
org.springframework.boot.logging.log4j2.Log4J2LoggingSystem.Factory,\
org.springframework.boot.logging.java.JavaLoggingSystem.Factory
# PropertySource Loaders
org.springframework.boot.env.PropertySourceLoader=\
org.springframework.boot.env.PropertiesPropertySourceLoader,\
org.springframework.boot.env.YamlPropertySourceLoader
# ConfigData Location Resolvers
org.springframework.boot.context.config.ConfigDataLocationResolver=\
org.springframework.boot.context.config.ConfigTreeConfigDataLocationResolver,\
org.springframework.boot.context.config.StandardConfigDataLocationResolver
Spring SPIIn , put all the configurations into a fixed file, saving you the trouble of configuring a lot of files. As for the extended configuration of multiple interfaces, whether it is better to use one file or each separate file is a matter of opinion.
Spring的SPIAlthough it belongs to it spring-framework(core), it is mainly used in spring boot...
Spring SPIThere are also ClassPathmultiple spring.factoriesfiles in the support, and classpaththese spring.factoriesfiles will be loaded in sequence according to the order of loading and added to one ArrayList. Since there is no alias, there is no concept of deduplication, and you can add as many as you have.
But because Springof SPIis mainly used in Spring Boot, and Spring Bootin ClassLoaderwill give priority to loading the files in the project instead of relying on the files in the package. So if you define a file in your project spring.factories, then the file in your project will be loaded first, Factoriesand the implementation class configured in the project spring.factorieswill also be ranked first.
如果我们要扩展某个接口的话，只需要在你的项目里新建一个META-INF/spring.factories文件，只添加你要的那个配置。
Dubbo SPI
Dubbo就是通过SPI机制加载所有的组件。不过，Dubbo并未使用 Java 原生的SPI机制，而是对其进行了增强，使其能够更好的满足需求。在Dubbo中，SPI是一个非常重要的模块。基于SPI，我们可以很容易的对Dubbo进行拓展。
Dubbo中实现了一套新的SPI 机制，功能更强大，也更复杂一些。相关逻辑被封装在了ExtensionLoader 类中，通过ExtensionLoader，我们可以加载指定的实现类。Dubbo SPI所需的配置文件需放置在META-INF/dubbo路径下，配置内容如下（以下demo来自dubbo官方文档）:
optimusPrime = org.apache.spi.OptimusPrime
bumblebee = org.apache.spi.Bumblebee
与Java SPI实现类配置不同，Dubbo SPI是通过键值对的方式进行配置，这样我们可以按需加载指定的实现类。另外在使用时还需要在接口上标注 @SPI注解。下面来演示Dubbo SPI的用法：
@SPI
public interface Robot {
void sayHello();
}
public class OptimusPrime implements Robot {
@Override
public void sayHello() {
System.out.println("Hello, I am Optimus Prime.");
}
}
public class Bumblebee implements Robot {
@Override
public void sayHello() {
System.out.println("Hello, I am Bumblebee.");
}
}
public class DubboSPITest {
@Test
public void sayHello() throws Exception {
ExtensionLoader<Robot> extensionLoader =
ExtensionLoader.getExtensionLoader(Robot.class);
Robot optimusPrime = extensionLoader.getExtension("optimusPrime");
optimusPrime.sayHello();
Robot bumblebee = extensionLoader.getExtension("bumblebee");
bumblebee.sayHello();
}
}
Dubbo SPI和JDK SPI最大的区别就在于支持“别名”，可以通过某个扩展点的别名来获取固定的扩展点。就像上面的例子中，我可以获取Robot多个SPI实现中别名为“optimusPrime”的实现，也可以获取别名为“bumblebee”的实现，这个功能非常有用！
通过@SPI注解的value属性，还可以默认一个“别名”的实现。比如在Dubbo中，默认的是Dubbo私有协议：dubbo protocol - dubbo://
来看看Dubbo中协议的接口：
@SPI("dubbo")
public interface Protocol {
......
}
在Protocol接口上，增加了一个@SPI注解，而注解的value值为Dubbo ，通过SPI获取实现时就会获取 Protocol SPI配置中别名为dubbo的那个实现，com.alibaba.dubbo.rpc.Protocol文件如下：
filter=com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper
listener=com.alibaba.dubbo.rpc.protocol.ProtocolListenerWrapper
mock=com.alibaba.dubbo.rpc.support.MockProtocol
dubbo=com.alibaba.dubbo.rpc.protocol.dubbo.DubboProtocol
injvm=com.alibaba.dubbo.rpc.protocol.injvm.InjvmProtocol
rmi=com.alibaba.dubbo.rpc.protocol.rmi.RmiProtocol
hessian=com.alibaba.dubbo.rpc.protocol.hessian.HessianProtocol
com.alibaba.dubbo.rpc.protocol.http.HttpProtocol
com.alibaba.dubbo.rpc.protocol.webservice.WebServiceProtocol
thrift=com.alibaba.dubbo.rpc.protocol.thrift.ThriftProtocol
memcached=com.alibaba.dubbo.rpc.protocol.memcached.MemcachedProtocol
redis=com.alibaba.dubbo.rpc.protocol.redis.RedisProtocol
rest=com.alibaba.dubbo.rpc.protocol.rest.RestProtocol
registry=com.alibaba.dubbo.registry.integration.RegistryProtocol
qos=com.alibaba.dubbo.qos.protocol.QosProtocolWrapper
然后只需要通过getDefaultExtension，就可以获取到@SPI注解上value对应的那个扩展实现了。
SPI深入理解
API与SPI的区别
API（Application Programming Interface）和SPI（Service Provider Interface）是两种不同的概念，API是软件组件之间的接口规范，用于定义交互方式和通信协议，以便于开发者使用和集成组件。而SPI是一种服务发现机制，用于动态加载和扩展应用程序中的服务提供者，允许通过插件式的方式添加和替换功能实现。API是软件开发中常见的概念，而SPI则是特定于服务发现和扩展的机制。
API（应用程序编程接口）：
API是一组定义了软件组件之间交互方式和通信协议的接口。
API提供了一系列的函数、方法、类、协议等，用于让开发者能够与某个软件库、框架或平台进行交互。
API定义了外部组件与提供者之间的约定和规范，以便于开发者可以使用和集成这些组件来实现特定的功能。
API通常由供应商或平台提供，并且在软件开发中广泛使用，以简化开发者的工作，提供特定功能和服务的访问途径。
SPI（服务提供者接口）：
SPI是一种服务发现机制，用于在运行时动态加载和扩展应用程序中的服务提供者。
SPI允许开发者定义服务接口，然后通过服务提供者实现该接口，并在运行时通过SPI机制动态发现和加载实现。
SPI通过在类路径下的META-INF/services目录中的配置文件中指定实现类的方式，使得应用程序可以通过插件式的方式添加、替换和扩展功能。
SPI提供了一种松耦合的方式，允许应用程序在不修改源代码的情况下，通过添加新的服务提供者实现来扩展功能。
ServiceLoader
//ServiceLoader实现了Iterable接口，可以遍历所有的服务实现者
public final class ServiceLoader<S>
implements Iterable<S>
{
//查找配置文件的目录
private static final String PREFIX = "META-INF/services/";
//表示要被加载的服务的类或接口
private final Class<S> service;
//这个ClassLoader用来定位，加载，实例化服务提供者
private final ClassLoader loader;
// 访问控制上下文
private final AccessControlContext acc;
// 缓存已经被实例化的服务提供者，按照实例化的顺序存储
private LinkedHashMap<String,S> providers = new LinkedHashMap<>();
// 迭代器
private LazyIterator lookupIterator;
//重新加载，就相当于重新创建ServiceLoader了，用于新的服务提供者安装到正在运行的Java虚拟机中的情况。
public void reload() {
//清空缓存中所有已实例化的服务提供者
providers.clear();
//新建一个迭代器，该迭代器会从头查找和实例化服务提供者
lookupIterator = new LazyIterator(service, loader);
}
//私有构造器
//使用指定的类加载器和服务创建服务加载器
//如果没有指定类加载器，使用系统类加载器，就是应用类加载器。
private ServiceLoader(Class<S> svc, ClassLoader cl) {
service = Objects.requireNonNull(svc, "Service interface cannot be null");
loader = (cl == null) ? ClassLoader.getSystemClassLoader() : cl;
acc = (System.getSecurityManager() != null) ? AccessController.getContext() : null;
reload();
}
//解析失败处理的方法
private static void fail(Class<?> service, String msg, Throwable cause)
throws ServiceConfigurationError
{
throw new ServiceConfigurationError(service.getName() + ": " + msg,
cause);
}
private static void fail(Class<?> service, String msg)
throws ServiceConfigurationError
{
throw new ServiceConfigurationError(service.getName() + ": " + msg);
}
private static void fail(Class<?> service, URL u, int line, String msg)
throws ServiceConfigurationError
{
fail(service, u + ":" + line + ": " + msg);
}
//解析服务提供者配置文件中的一行
//首先去掉注释校验，然后保存
//返回下一行行号
//重复的配置项和已经被实例化的配置项不会被保存
private int parseLine(Class<?> service, URL u, BufferedReader r, int lc,
List<String> names)
throws IOException, ServiceConfigurationError
{
//读取一行
String ln = r.readLine();
if (ln == null) {
return -1;
}
//#号代表注释行
int ci = ln.indexOf('#');
if (ci >= 0) ln = ln.substring(0, ci);
ln = ln.trim();
int n = ln.length();
if (n != 0) {
if ((ln.indexOf(' ') >= 0) || (ln.indexOf('\t') >= 0))
fail(service, u, lc, "Illegal configuration-file syntax");
int cp = ln.codePointAt(0);
if (!Character.isJavaIdentifierStart(cp))
fail(service, u, lc, "Illegal provider-class name: " + ln);
for (int i = Character.charCount(cp); i < n; i += Character.charCount(cp)) {
cp = ln.codePointAt(i);
if (!Character.isJavaIdentifierPart(cp) && (cp != '.'))
fail(service, u, lc, "Illegal provider-class name: " + ln);
}
if (!providers.containsKey(ln) && !names.contains(ln))
names.add(ln);
}
return lc + 1;
}
//解析配置文件，解析指定的url配置文件
//使用parseLine方法进行解析，未被实例化的服务提供者会被保存到缓存中去
private Iterator<String> parse(Class<?> service, URL u)
throws ServiceConfigurationError
{
InputStream in = null;
BufferedReader r = null;
ArrayList<String> names = new ArrayList<>();
try {
in = u.openStream();
r = new BufferedReader(new InputStreamReader(in, "utf-8"));
int lc = 1;
while ((lc = parseLine(service, u, r, lc, names)) >= 0);
}
return names.iterator();
}
//服务提供者查找的迭代器
private class LazyIterator
implements Iterator<S>
{
Class<S> service;//服务提供者接口
ClassLoader loader;//类加载器
Enumeration<URL> configs = null;//保存实现类的url
Iterator<String> pending = null;//保存实现类的全名
String nextName = null;//迭代器中下一个实现类的全名
private LazyIterator(Class<S> service, ClassLoader loader) {
this.service = service;
this.loader = loader;
}
private boolean hasNextService() {
if (nextName != null) {
return true;
}
if (configs == null) {
try {
String fullName = PREFIX + service.getName();
if (loader == null)
configs = ClassLoader.getSystemResources(fullName);
else
configs = loader.getResources(fullName);
}
}
while ((pending == null) || !pending.hasNext()) {
if (!configs.hasMoreElements()) {
return false;
}
pending = parse(service, configs.nextElement());
}
nextName = pending.next();
return true;
}
private S nextService() {
if (!hasNextService())
throw new NoSuchElementException();
String cn = nextName;
nextName = null;
Class<?> c = null;
try {
c = Class.forName(cn, false, loader);
}
if (!service.isAssignableFrom(c)) {
fail(service, "Provider " + cn + " not a subtype");
}
try {
S p = service.cast(c.newInstance());
providers.put(cn, p);
return p;
}
}
public boolean hasNext() {
if (acc == null) {
return hasNextService();
} else {
PrivilegedAction<Boolean> action = new PrivilegedAction<Boolean>() {
public Boolean run() {
return hasNextService(); }
};
return AccessController.doPrivileged(action, acc);
}
}
public S next() {
if (acc == null) {
return nextService();
} else {
PrivilegedAction<S> action = new PrivilegedAction<S>() {
public S run() {
return nextService(); }
};
return AccessController.doPrivileged(action, acc);
}
}
public void remove() {
throw new UnsupportedOperationException();
}
}
//获取迭代器
//返回遍历服务提供者的迭代器
//以懒加载的方式加载可用的服务提供者
//懒加载的实现是：解析配置文件和实例化服务提供者的工作由迭代器本身完成
public Iterator<S> iterator() {
return new Iterator<S>() {
//按照实例化顺序返回已经缓存的服务提供者实例
Iterator<Map.Entry<String,S>> knownProviders
= providers.entrySet().iterator();
public boolean hasNext() {
if (knownProviders.hasNext())
return true;
return lookupIterator.hasNext();
}
public S next() {
if (knownProviders.hasNext())
return knownProviders.next().getValue();
return lookupIterator.next();
}
public void remove() {
throw new UnsupportedOperationException();
}
};
}
//为指定的服务使用指定的类加载器来创建一个ServiceLoader
public static <S> ServiceLoader<S> load(Class<S> service,
ClassLoader loader)
{
return new ServiceLoader<>(service, loader);
}
//使用线程上下文的类加载器来创建ServiceLoader
public static <S> ServiceLoader<S> load(Class<S> service) {
ClassLoader cl = Thread.currentThread().getContextClassLoader();
return ServiceLoader.load(service, cl);
}
//使用扩展类加载器为指定的服务创建ServiceLoader
//只能找到并加载已经安装到当前Java虚拟机中的服务提供者，应用程序类路径中的服务提供者将被忽略
public static <S> ServiceLoader<S> loadInstalled(Class<S> service) {
ClassLoader cl = ClassLoader.getSystemClassLoader();
ClassLoader prev = null;
while (cl != null) {
prev = cl;
cl = cl.getParent();
}
return ServiceLoader.load(service, prev);
}
public String toString() {
return "java.util.ServiceLoader[" + service.getName() + "]";
}
}
首先，ServiceLoader实现了Iterable接口，所以它有迭代器的属性，这里主要都是实现了迭代器的hasNext和next方法。这里主要都是调用的lookupIterator的相应hasNext和next方法，lookupIterator是懒加载迭代器。
其次，LazyIterator中的hasNext方法，静态变量PREFIX就是META-INF/services/目录，这也就是为什么需要在classpath下的META-INF/services/目录里创建一个以服务接口命名的文件。
最后，通过反射方法Class.forName()加载类对象，并用newInstance方法将类实例化，并把实例化后的类缓存到providers对象中，(LinkedHashMap<String,S>类型）然后返回实例对象。
So we can see ServiceLoaderthat instead of instantiation, we read the specific implementation in the configuration file and instantiate it. Instead, when the iterator is used to traverse, the corresponding configuration file will be loaded for parsing. When the hasNext method is called, the configuration file will be loaded for parsing. When the method is called, it will be instantiated and cached next. All configuration files will only be loaded once, and the service provider will only be instantiated once. Reload configuration files can be used reload.
Comprehensive comparison of JDK SPI, Spring SPI, and Dubbo SPI
| JDK SPI | DUBBO SPI | Spring SPI | |
|---|---|---|---|
| file format | A separate file for each extension point | A separate file for each extension point | All extension points in one file |
| Get a fixed implementation | Not supported, can only get all implementations in order | With the concept of "alias", you can get a fixed implementation of the extension point through the name, which is very convenient to cooperate with Dubbo SPI annotations | Not supported, all implementations can only be obtained sequentially. However, since the Spring Boot ClassLoader will load the files in the user code first, it can be guaranteed that the user-defined spring.factoires file is the first, and the custom extension can be fixed by obtaining the first factory. |
| other | none | Supports dependency injection inside Dubbo, distinguishes Dubbo built-in SPI and external SPI through the directory, and loads the internal first to ensure the highest priority of the internal | none |
| Documentation Completeness | Articles & third-party data are rich enough | Documentation & third-party information is rich enough | The documentation is not rich enough, but due to the small number of functions, it is very simple to use |
| IDE support | none | none | IDEA perfectly supports it, with syntax hints |