What is a microkernel architecture?
The microkernel is a typical architecture model, which is different from the ordinary design model. The architecture model is a high-level model used to describe the system-level structural composition, mutual relationships and related constraints. Microkernel architecture is widely used in open source frameworks. For example, the common ShardingSphere and Dubbo have implemented their own microkernel architecture. So, before introducing what is a microkernel architecture, we need to explain the reasons why these open source frameworks use microkernel architecture.
Why use microkernel architecture?
The microkernel architecture is essentially to improve the scalability of the system. The so-called scalability refers to the flexibility that the system has when undergoing unavoidable changes and the ability to balance the cost of providing such flexibility. In other words, when adding a new business to the system, there is no need to change the original components, and only need to close the new business in a new component to complete the upgrade of the overall business. We believe that such a system has more advantages. Good scalability.
As far as architecture design is concerned, scalability is an eternal topic in software design. To achieve system scalability, one way of thinking is to provide a pluggable mechanism to respond to changes that occur. When an existing component in the system does not meet the requirements, we can implement a new component to replace it, and the whole process should be unaware of the operation of the system. We can also complete this new and old as needed. Replacement of components.
For example, in the distributed primary key function provided in ShardingSphere, there may be many implementations of distributed primary key, and the scalability at this point is that we can use any new distributed primary key implementation to replace the original implementation , Without the need to make any changes to the business code that depends on the distributed primary key.
The micro-kernel architecture model provides architectural design support for this idea of achieving scalability. ShardingSphere achieves a high degree of scalability based on the micro-kernel architecture. Before introducing how to implement the micro-kernel architecture, we first briefly explain the specific structure and basic principles of the micro-kernel architecture.
What is a microkernel architecture?
In terms of composition structure, the microkernel architecture consists of two components: the kernel system and the plug-in. The kernel system here usually provides the minimum set of functions required for system operation, while the plug-in is an independent component that contains various custom business codes to enhance or extend additional business capabilities to the kernel system. In ShardingSphere, the aforementioned distributed primary key is the plug-in, and the runtime environment of ShardingSphere constitutes the kernel system.
So what exactly does the plugin here refer to? This requires us to clarify two concepts. One concept is the API that we often talk about, which is the interface exposed by the system. Another concept is SPI (Service Provider Interface), which is the extension point of the plug-in itself. As far as the relationship between the two is concerned, API is for business developers, while SPI is for framework developers. Together, the two constitute ShardingSphere itself.
The pluggable implementation mechanism is simple to say, but not easy to do. We need to consider two aspects. On the one hand, we need to sort out the changes in the system and abstract them into multiple SPI extension points. On the other hand, when we implement these SPI extension points, we need to build a specific implementation that can support this pluggable mechanism, thereby providing a SPI runtime environment.
How to realize the microkernel architecture?
In fact, JDK has provided us with a way to implement the micro-kernel architecture, which is JDK SPI. This implementation method puts forward some development and configuration specifications for how to design and implement SPI. ShardingSphere and Dubbo use this specification, but they are enhanced and optimized on this basis. So to understand how to implement the microkernel architecture, we might as well take a look at the working principle of JDK SPI.
JDK SPI
SPI (Service Provider Interface) is mainly a technology used by framework developers. For example, when using the Java language we use to access the database java.sql.Driverinterfaces, different database products underlying agreement, provided java.sql.Driverimplementation is different, in the development of java.sql.Driverinterface, the developers do not know what the final user database will be used, in which case you can use the Java SPI mechanisms in actual operation, as java.sql.Driverthe interface to find the specific implementation.
Below we demonstrate the use of JDK SPI through a simple example:
-
First, we define an interface for generating id keys to simulate id generation
public interface IdGenerator {
/**
* 生成id
* @return
*/
String generateId();
}
-
Then create two interface implementation classes to simulate the generation of uuid and serial id respectively
public class UuidGenerator implements IdGenerator {
@Override
public String generateId() {
return UUID.randomUUID().toString();
}
}
public class SequenceIdGenerator implements IdGenerator {
private final AtomicLong atomicId = new AtomicLong(100L);
@Override
public String generateId() {
long leastId = this.atomicId.incrementAndGet();
return String.valueOf(leastId);
}
}
-
resources/META-INF/servicesAdd acom.github.jianzh5.spi.IdGeneratorfile named under the project directory , this is the configuration file that JDK SPI needs to read, the content is as follows:
com.github.jianzh5.spi.impl.UuidGenerator
com.github.jianzh5.spi.impl.SequenceIdGenerator
-
Create the main method, let it load the above configuration file, create all the instances of the IdGenerator interface implementation, and execute the method of generating id.
public class GeneratorMain {
public static void main(String[] args) {
ServiceLoader<IdGenerator> serviceLoader = ServiceLoader.load(IdGenerator.class);
Iterator<IdGenerator> iterator = serviceLoader.iterator();
while(iterator.hasNext()){
IdGenerator generator = iterator.next();
String id = generator.generateId();
System.out.println(generator.getClass().getName() + " >>id:" + id);
}
}
}
-
The execution results are as follows:
JDK SPI source code analysis
Through the above example, we can see that the entry method of the JDK SPI is the ServiceLoader.load() method. In this method, it will first try to get the currently used ClassLoader, and then call the reload() method. The calling relationship is shown in the following figure:
In the reload() method, the provider cache (a collection of LinkedHashMap type) is first cleaned up. The cache is used to record the implementation objects created by ServiceLoader, where Key is the complete class name of the implementation class, and Value is the object of the implementation class. Then create a LazyIterator iterator to read the SPI configuration file and instantiate the implementation class object.
public void reload() {
providers.clear();
lookupIterator = new LazyIterator(service, loader);
}
In the previous example, main () method is used to call the iterator underlying ServiceLoader.LazyIteratorimplementation. Iterator interface has two key methods: hasNext()methods and next()methods. LazyIterator here in the next()method call is the final nextService()method, hasNext()the method is the final call hasNextService()method, we look at hasNextService()specific implementation of the method:
private static final String PREFIX = "META-INF/services/";
Enumeration<URL> configs = null;
Iterator<String> pending = null;
String nextName = null;
private boolean hasNextService() {
if (nextName != null) {
return true;
}
if (configs == null) {
try {
//META-INF/services/com.github.jianzh5.spi.IdGenerator
String fullName = PREFIX + service.getName();
if (loader == null)
configs = ClassLoader.getSystemResources(fullName);
else
configs = loader.getResources(fullName);
} catch (IOException x) {
fail(service, "Error locating configuration files", x);
}
}
// 按行SPI遍历配置文件的内容
while ((pending == null) || !pending.hasNext()) {
if (!configs.hasMoreElements()) {
return false;
}
// 解析配置文件
pending = parse(service, configs.nextElement());
}
// 更新 nextName字段
nextName = pending.next();
return true;
}
After parsing the SPI configuration file in the hasNextService() method, let’s look at the LazyIterator.nextService() method, which is "responsible for instantiating the implementation class read by the hasNextService() method" , which will place the instantiated object in Cached in providers collection, the core implementation is as follows:
private S nextService() {
String cn = nextName;
nextName = null;
// 加载 nextName字段指定的类
Class<?> c = Class.forName(cn, false, loader);
if (!service.isAssignableFrom(c)) { // 检测类型
fail(service, "Provider " + cn + " not a subtype");
}
S p = service.cast(c.newInstance()); // 创建实现类的对象
providers.put(cn, p); // 将实现类名称以及相应实例对象添加到缓存
return p;
}
The above is the underlying implementation of the iterator used in the main() method. Finally, we'll look at the main () method using a ServiceLoader.iterator()method to get the iterator is how to achieve this iterator is dependent on an anonymous inner class LazyIterator implemented the core to achieve the following:
public Iterator<S> iterator() {
return new Iterator<S>() {
// knownProviders用来迭代providers缓存
Iterator<Map.Entry<String,S>> knownProviders
= providers.entrySet().iterator();
public boolean hasNext() {
// 先走查询缓存，缓存查询失败，再通过LazyIterator加载
if (knownProviders.hasNext())
return true;
return lookupIterator.hasNext();
}
public S next() {
// 先走查询缓存，缓存查询失败，再通过 LazyIterator加载
if (knownProviders.hasNext())
return knownProviders.next().getValue();
return lookupIterator.next();
}
// 省略remove()方法
};
}
Application of JDK SPI in JDBC
After understanding the principle of JDK SPI implementation, let's look at how JDBC uses the JDK SPI mechanism to load the implementation classes of different database vendors in practice.
JDK only defines an java.sql.Driverinterface, and implementation by different database vendors to offer. Here we take the JDBC implementation package provided by MySQL as an example for analysis.
In the META-INF/services directory in the mysql-connector-java-*.jar package, there is a java.sql.Driver file with only one line, as shown below:
com.mysql.cj.jdbc.Driver
In use mysql-connector-java-*.jarwhen connected to MySQL database package, we will use the following statement creates a database connection:
String url = "jdbc:xxx://xxx:xxx/xxx";
Connection conn = DriverManager.getConnection(url, username, pwd);
"DriverManager is a database driver manager provided by JDK" , the code snippets are as follows:
static {
loadInitialDrivers();
println("JDBC DriverManager initialized");
}
Call getConnection()time method, the DriverManager class is loaded Java virtual machine, and trigger the execution of analytical static code block; in the loadInitialDrivers()process by the scanning Classpath JDK SPI java.sql.Driverinterface class and instance, core implementation is as follows:
private static void loadInitialDrivers() {
String drivers = System.getProperty("jdbc.drivers")
// 使用 JDK SPI机制加载所有 java.sql.Driver实现类
ServiceLoader<Driver> loadedDrivers =
ServiceLoader.load(Driver.class);
Iterator<Driver> driversIterator = loadedDrivers.iterator();
while(driversIterator.hasNext()) {
driversIterator.next();
}
String[] driversList = drivers.split(":");
for (String aDriver : driversList) { // 初始化Driver实现类
Class.forName(aDriver, true,
ClassLoader.getSystemClassLoader());
}
}
MySQL provided in com.mysql.cj.jdbc.Driverthe implementation class, there is a static same static block of code, the code creates an com.mysql.cj.jdbc.Driverobject and to register DriverManager.registeredDriversthe set (a CopyOnWriteArrayList type), as follows:
static {
java.sql.DriverManager.registerDriver(new Driver());
}
In getConnection(), DriverManager registeredDrivers acquisition method from the corresponding set of object creation Driver Connection, core implementation is as follows:
private static Connection getConnection(String url, java.util.Properties info, Class<?> caller) throws SQLException {
// 省略 try/catch代码块以及权限处理逻辑
for(DriverInfo aDriver : registeredDrivers) {
Connection con = aDriver.driver.connect(url, info);
return con;
}
}
summary
In this article, we described in detail some basic concepts of the microkernel architecture and started with an example, introduced the basic use of the SPI mechanism provided by the JDK, and then deeply analyzed the core principles and underlying implementation of the JDK SPI, and conducted an in-depth analysis of its source code. Finally, we take the JDBC implementation provided by MySQL as an example to analyze the use of JDK SPI in practice.
Mastering the SPI mechanism of the JDK is equivalent to mastering the core of the microkernel architecture. The above, I hope it will help you!