Starting today, we will enter a new module that ShardingSphere execution engine (ExecuteEngine) module. Execution engine is responsible for obtaining from the routing engine and rewrite the generated SQL and complete execution in a specific database. ShardingSphere execution engine is the core module, we will take a relatively long length of its full introduction.
1. The overall structure of the execution engine
Before explaining the specific code, we still from "Analysis of ShardingSphere Source Routing Engine (g)" PreparedQueryShardingEngine and SimpleQueryShardingEngine described in these two classes starting to see the entrance they are referenced. We have found the following shard method ShardingPreparedStatement class, shardingEngine here is PreparedQueryShardingEngine:
private void shard() {
sqlRouteResult = shardingEngine.shard(sql, getParameters());
}
Then, we also find the following method in ShardingStatement, it is clear here uses SimpleQueryShardingEngine:
private void shard(final String sql) {
ShardingRuntimeContext runtimeContext = connection.getRuntimeContext();
SimpleQueryShardingEngine shardingEngine = new SimpleQueryShardingEngine(runtimeContext.getRule(), runtimeContext.getProps(), runtimeContext.getMetaData(), runtimeContext.getParseEngine());
sqlRouteResult = shardingEngine.shard(sql, Collections.emptyList());
}
Speaking from the design mode, ShardingStatement and ShardingPreparedStatement is actually very typical facade (Facade) class, the class entry with SQL routing and execution are integrated together. Therefore, we can find the entrance of the relevant class execution engine in both the facade class. By reading the source code, we find that there is a StatementExecutor in ShardingStatement, whereas there PreparedStatementExecutor and BatchPreparedStatementExecutor in ShardingPreparedStatement, these classes are to (actuators) the end of the Executor, obviously we are looking for SQL execution entry class engine.
We found that the above three are located Executor sharding-jdbc-core projects. But there is another and sharding-core-route and sharding-core-rewrite tied sharding-core-execute the project, from the naming point of view, this project should be related to the execution engine. Sure enough, we found ShardingExecuteEngine class in this project, which is the entry class slice execution engine. Then, we were found SQLExecuteTemplate and SQLExecutePrepareTemplate classes, which is a typical SQL execution template class.
According to date knowledge of ShardingSphere coding style, it is conceivable, in the hierarchy, ShardingExecuteEngine is the underlying object, SQLExecuteTemplate should depend on ShardingExecuteEngine. And StatementExecutor, PreparedStatementExecutor and BatchPreparedStatementExecutor belonging to the upper target, should depend on SQLExecuteTemplate. We read reference relationship before these core classes by simply confirms this conjecture.
Based on the above analysis, we can give the following SQL execution engine overall configuration diagram, which portion is located above the straight line sharding-core-execute the project, are the underlying components. The following sections positioned linearly sharding-jdbc-core belonging to the upper layer assembly:
We also see in the picture above SQLExecuteCallback and SQLExecutePrepareCallback, obviously, their role is to complete the callback process SQL execution, which is a very typical scalable approach.
2. Kshrdidagekshechuteadagine
By convention, we began to cut into the ground floor ShardingExecuteEngine. With different routing and rewriting engine, ShardingExecuteEngine ShardingSphere is the only one execution engine, it is designed directly as a class and interface. This class includes the variables and the constructor as follows:
private final ShardingExecutorService shardingExecutorService;
private ListeningExecutorService executorService;
public ShardingExecuteEngine(final int executorSize) {
shardingExecutorService = new ShardingExecutorService(executorSize);
executorService = shardingExecutorService.getExecutorService();
}
There are two variables to see ExecutorService end, it is clear from the name we can see that they are to perform service, and the JDK java.util.concurrent.ExecutorService similar. Wherein ListeningExecutorService google kit from guava, ShardingExecutorService custom class, and contains ListeningExecutorService ShardingExecutorService build process. We first look at ShardingExecutorService.
We found ShardingExecutorService contains a JDK of ExecutorService, as shown in its creation process, as used herein, it is a common practice in the JDK:
private ExecutorService getExecutorService(final int executorSize, final String nameFormat) {
ThreadFactory shardingThreadFactory = ShardingThreadFactoryBuilder.build(nameFormat);
return 0 == executorSize ? Executors.newCachedThreadPool(shardingThreadFactory) : Executors.newFixedThreadPool(executorSize, shardingThreadFactory);
}
Since the JDK common thread pool returned Future single function, so guava provides ListeningExecutorService decorate it. We can do by ListeningExecutorService ExecutorService layer of packaging, returns a ListenableFuture instance, while ListenableFuture is inherited from the Future, we extended a listener addListener method, so that when the task is completed it will automatically execute the callback method. ListeningExecutorService build process is as follows:
public ShardingExecutorService(final int executorSize, final String nameFormat) {
executorService = MoreExecutors.listeningDecorator(getExecutorService(executorSize, nameFormat));
MoreExecutors.addDelayedShutdownHook(executorService, 60, TimeUnit.SECONDS);
}
After the clear actuator ExecutorService, we returned ShardingExecuteEngine class, which provides a groupExecute method as the entry, as shown below (which are more parameters, also are listed separately for a moment):
/**
* @Param inputGroups : Enter the group
* @Param firstCallback : for the first time slice callback
* @Param callback : callback slices
* @Param Serial : whether to use a multi-threaded execution
* @Param <the I> : Input Value Type
* @Param <O> : Return Value Type
* @Return the results
* @Throws SQLException : throw an exception
*/
public <I, O> List<O> groupExecute(
final Collection<ShardingExecuteGroup<I>> inputGroups, final ShardingGroupExecuteCallback<I, O> firstCallback, final ShardingGroupExecuteCallback<I, O> callback, final boolean serial)
throws SQLException {
if (inputGroups.isEmpty()) {
return Collections.emptyList();
}
return serial ? serialExecute(inputGroups, firstCallback, callback) : parallelExecute(inputGroups, firstCallback, callback);
}
Here is a list of ShardingExecuteGroup object is actually input information includes, defined as follows:
public final class ShardingExecuteGroup<T> {
private final List<T> inputs;
}
The input to the method is a ShardingExecuteGroup groupExecute collection. Is determined by the input parameters serial, the above code and flow were turned serialExecute parallelExecute two code branches.
Let us first look serialExecute method, as the name implies, the method is for serial execution of the scenario, which is defined as follows:
private <I, O> List<O> serialExecute(final Collection<ShardingExecuteGroup<I>> inputGroups, final ShardingGroupExecuteCallback<I, O> firstCallback,
final ShardingGroupExecuteCallback<I, O> callback) throws SQLException {
Iterator<ShardingExecuteGroup<I>> inputGroupsIterator = inputGroups.iterator();
ShardingExecuteGroup<I> firstInputs = inputGroupsIterator.next();
List<O> result = new LinkedList<>(syncGroupExecute(firstInputs, null == firstCallback ? callback : firstCallback));
for (ShardingExecuteGroup<I> each : Lists.newArrayList(inputGroupsIterator)) {
result.addAll(syncGroupExecute(each, callback));
}
return result;
}
The basic process is to obtain the above code ShardingExecuteGroup first input through a first synchronous execution completion callback firstCallback syncGroupExecute. Then the remaining ShardingExecuteGroup, one by one through the implementation of syncGroupExecute callback callback. Here syncGroupExecute methods are as follows:
private <I, O> Collection<O> syncGroupExecute(final ShardingExecuteGroup<I> executeGroup, final ShardingGroupExecuteCallback<I, O> callback) throws SQLException {
return callback.execute(executeGroup.getInputs(), true, ShardingExecuteDataMap.getDataMap());
}
We see the process of synchronizing execution actually handed over ShardingGroupExecuteCallback, ShardingGroupExecuteCallback interfaces are defined as follows:
public interface ShardingGroupExecuteCallback<I, O> {
Collection<O> execute(Collection<I> inputs, boolean isTrunkThread, Map<String, Object> shardingExecuteDataMap) throws SQLException;
}
ShardingExecuteDataMap here is equivalent to a data dictionary for the implementation of SQL, these data dictionary stored in ThreadLocal, ensuring thread safety. We can get the corresponding DataMap objects based on the current thread of execution.
Thus, with respect to the flow introduced over serial execution, then we look parallelExecute method performed in parallel, as follows:
private <I, O> List<O> parallelExecute(final Collection<ShardingExecuteGroup<I>> inputGroups, final ShardingGroupExecuteCallback<I, O> firstCallback, final ShardingGroupExecuteCallback<I, O> callback) throws SQLException {
Iterator<ShardingExecuteGroup<I>> inputGroupsIterator = inputGroups.iterator();
ShardingExecuteGroup<I> firstInputs = inputGroupsIterator.next();
Collection<ListenableFuture<Collection<O>>> restResultFutures = asyncGroupExecute(Lists.newArrayList(inputGroupsIterator), callback);
return getGroupResults(syncGroupExecute(firstInputs, null == firstCallback ? callback : firstCallback), restResultFutures);
}
Notice that there are an asynchronous method asyncGroupExecute, passing parameters is a List <ShardingExecuteGroup>, as follows:
private <I, O> Collection<ListenableFuture<Collection<O>>> asyncGroupExecute(final List<ShardingExecuteGroup<I>> inputGroups, final ShardingGroupExecuteCallback<I, O> callback) {
Collection<ListenableFuture<Collection<O>>> result = new LinkedList<>();
for (ShardingExecuteGroup<I> each : inputGroups) {
result.add(asyncGroupExecute(each, callback));
}
return result;
}
Then for each incoming ShardingExecuteGroup, call asyncGroupExecute method with the same name again:
private <I, O> ListenableFuture<Collection<O>> asyncGroupExecute(final ShardingExecuteGroup<I> inputGroup, final ShardingGroupExecuteCallback<I, O> callback) {
final Map<String, Object> dataMap = ShardingExecuteDataMap.getDataMap();
return executorService.submit(new Callable<Collection<O>>() {
@Override
public Collection<O> call() throws SQLException {
return callback.execute(inputGroup.getInputs(), false, dataMap);
}
});
}
Obviously, as asynchronous execution method, where it will use ListeningExecutorService to submit an asynchronous task and returns a ListenableFuture, but this task is performed asynchronously specific callback.
Finally, we look at the last sentence parallelExecute method that calls getGroupResults method to obtain the results as follows:
private <O> List<O> getGroupResults(final Collection<O> firstResults, final Collection<ListenableFuture<Collection<O>>> restFutures) throws SQLException {
List<O> result = new LinkedList<>(firstResults);
for (ListenableFuture<Collection<O>> each : restFutures) {
try {
result.addAll(each.get());
} catch (final InterruptedException | ExecutionException ex) {
return throwException(ex);
}
}
return result;
}
Future students are familiar with the use of the above code should not be unfamiliar, we traverse ListenableFuture, then mobilize its way to synchronize wait for its return to get the results. Finally, when all the results are obtained and assembled into a list of results returned. The wording is very common when using Future.
Looking back, both serialExecute method or methods parallelExecute will get from ShardingExecuteGroup the first firstInputs elements execution, then the rest of the re-synchronous or asynchronous execution, the purpose of this design is to make the first task by the current thread execution, so as not to waste a thread. It is also a multi-threaded application is a skill.
So far, on the introduction ShardingExecuteEngine class came to an end. As the execution engine, doing things ShardingExecuteEngine is to provide a multi-threaded execution environment. In essence, ShardingExecuteEngine not do anything business-related, but provides multi-threaded execution environment, the implementation of an incoming callback function, which is a very clever design. The same technique can also be seen in many other places and other open source frameworks such as Spring.
I can focus more public numbers: Programmer's transition to an architect.
