Recent research ShardingSphere (https://shardingsphere.apache.org/index_zh.html) source code, ready to begin a series to explain this sub-library sub-table design and implementation of middleware principle. Our first concern is the SQL engine parses the content part, this part is also very much we should spend a few articles to explain them, this is the first part of this series.
Want to read the source code, the system must first find the entrance, we can use the sample code from this ShardingSphere official website to make this clear:
// Configure real data source
the Map <String, the DataSource> = new new dataSourceMap the HashMap <> ();
// arranged a first data source
of BasicDataSource DataSource1 of BasicDataSource new new = ();
dataSource1.setDriverClassName ( "com.mysql.jdbc.Driver" );
dataSource1.setUrl ( "JDBC: MySQL: // localhost: 3306 / DS0");
dataSource1.setUsername ( "the root");
dataSource1.setPassword ( "");
dataSourceMap.put ( "DS0", DataSource1);
/ / configuration of the second data sources
of BasicDataSource DataSource2 of BasicDataSource new new = ();
dataSource2.setDriverClassName ( "com.mysql.jdbc.Driver");
dataSource2.setUrl ( "JDBC: MySQL: // localhost: 3306 / DS1");
DataSource2 .setUsername("root");
dataSource2.setPassword("");
dataSourceMap.put("ds1", dataSource2);
// 配置Order表规则
TableRuleConfiguration orderTableRuleConfig = new TableRuleConfiguration("t_order","ds${0..1}.t_order${0..1}");
// 配置分库 + 分表策略
orderTableRuleConfig.setDatabaseShardingStrategyConfig(new InlineShardingStrategyConfiguration("user_id", "ds${user_id % 2}"));
orderTableRuleConfig.setTableShardingStrategyConfig(new InlineShardingStrategyConfiguration("order_id", "t_order${order_id % 2}"));
// 配置分片规则
ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
. shardingRuleConfig.getTableRuleConfigs () the Add (orderTableRuleConfig);
// omitted arranged order_item sheet rules ...
// ...
// get the data source object
DataSource dataSource = ShardingDataSourceFactory.createDataSource (dataSourceMap, shardingRuleConfig , new Properties ());
Here we constructed several data sources, plus sub-libraries, sub-table strategy, and fragmentation rules, then get the data source DataSource by ShardingDataSourceFactory, apparently DataSource is our entrance. Around the DataSource, the tracking code by calling a link, we can get based layer structure shown in FIG follows:
About code examples, and this picture, we will often see in a later article, today we focus on the entire link in the bottom of the object, that figure SQLParseEngine. On the one hand, the process of creating the final DataSource created SQLParseEngine, on the other hand, ShardingRouter responsible for implementing the routing function also depends on SQLParseEngine. This is the whole ShardingSphere SQLParseEngine in charge of core SQL parsing.
AbstractRuntimeContext inlet is constructed SQLParseEngine constructed SQLParseEngine code is as follows:
protected AbstractRuntimeContext(final T rule, final Properties props, final DatabaseType databaseType) {
this.rule = rule;
this.props = new ShardingProperties(null == props ? new Properties() : props);
this.databaseType = databaseType;
executeEngine = new ShardingExecuteEngine(this.props.<Integer>getValue(ShardingPropertiesConstant.EXECUTOR_SIZE));
parseEngine = SQLParseEngineFactory.getSQLParseEngine(DatabaseTypes.getTrunkDatabaseTypeName(databaseType));
ConfigurationLogger.log(rule.getRuleConfiguration());
ConfigurationLogger.log(props);
}
Around SQLParseEngine, let's look at its factory class SQLParseEngineFactory, code is as follows:
public final class SQLParseEngineFactory {
private static final Map<String, SQLParseEngine> ENGINES = new ConcurrentHashMap<>();
public static SQLParseEngine getSQLParseEngine(final String databaseTypeName) {
if (ENGINES.containsKey(databaseTypeName)) {
return ENGINES.get(databaseTypeName);
}
synchronized (ENGINES) {
if (ENGINES.containsKey(databaseTypeName)) {
return ENGINES.get(databaseTypeName);
}
SQLParseEngine result = new SQLParseEngine(databaseTypeName);
ENGINES.put(databaseTypeName, result);
return result;
}
}
}
Here you can see a layer of caching to do, look at SQLParseEngine itself:
public final class SQLParseEngine {
private final String databaseTypeName;
private final SQLParseResultCache cache = new SQLParseResultCache();
public SQLStatement parse(final String sql, final boolean useCache) {
ParsingHook parsingHook = new SPIParsingHook();
parsingHook.start(sql);
try {
SQLStatement result = parse0(sql, useCache);
parsingHook.finishSuccess(result);
return result;
} catch (final Exception ex) {
parsingHook.finishFailure(ex);
throw ex;
}
}
private SQLStatement parse0(final String sql, final boolean useCache) {
if (useCache) {
Optional<SQLStatement> cachedSQLStatement = cache.getSQLStatement(sql);
if (cachedSQLStatement.isPresent()) {
return cachedSQLStatement.get();
}
}
SQLStatement result = new SQLParseKernel(ParseRuleRegistry.getInstance(), databaseTypeName, sql).parse();
if (useCache) {
cache.put(sql, result);
}
return result;
}
}
There are several points worth noting about SQLParseEngine. First, that hooks operating as a management system of ParsingHook, ParsingHook defined as follows:
public interface ParsingHook {
void start(String sql);
void finishSuccess(SQLStatement sqlStatement);
void finishFailure(Exception cause);
}
ShardingSphere achieve a series of ParsingHook, as shown in the following relationship based layer, in subsequent discussions we have time to ShardingSphere Linktrace OpenTracingParsingHook further expanded.
Secondly, there is also based on google Cache class construct SQLParseResultCache, to perform caching of the parsed SQLStatement.
Then, we found SQLParseEngine the real analytic work transferred to SQLParseKernel, in the class, found three core object is defined as the following:
private final SQLParserEngine parserEngine;
private final SQLSegmentsExtractorEngine extractorEngine;
private final SQLStatementFillerEngine fillerEngine;
Engine three separate classes three things completed, i.e., SQL parsing (SQLParser), SQLSegment extraction (SQLSegmentExtractor) and filling SQLStatement (SQLStatementFiller). Based on the description ShardingSphere official website, which is the core part of the new generation ShardingSphere SQL parsing engine, as shown in (ShardingSphere from official website) under its overall architecture of FIG.
We refer to SQLParseKernel achieve a better understanding of the structure of the above figure, SQLParseKernel parse method is as follows:
public SQLStatement parse() {
// use ANTLR4 resolve SQLAST , namely SQL abstract syntax tree
SQLAST ast = parserEngine.parse();
// extract AST in the Token , the package to the corresponding Segment , such as Table Segment , IndexSegment
Collection<SQLSegment> sqlSegments = extractorEngine.extract(ast);
Map<ParserRuleContext, Integer> parameterMarkerIndexes = ast.getParameterMarkerIndexes();
//填充SQLStatement并返回
return fillerEngine.fill(sqlSegments, parameterMarkerIndexes.size(), ast.getSqlStatementRule());
}
这样我们看到解析、提取、填充这三个步骤构成的整体流程已经完成，现在能够根据一条SQL语句解析出对应的SQLStatement对象供后续的ShardingRoute等对象进行使用。当然，围绕解析、提取、填充这三个步骤我们还有很多细节没有展开，今天的内容就到此为止，下一篇文章我们会更多关注这些细节。
更多内容可以关注我的公众号：程序员向架构师转型。
