This paper is mainly based on shared Spring Boot + Druid + MyBatis + ShardingSphere -JDBC sub-table and the sub-library separate read and write cases, the sample code, see item address sub-library sub-table .
1. Introduction to ShardingSphere-JDBC
1 、 ShardingSphere
In fact, ShardingSphere consists of 3 parts: ShardingSphere-JDBC, ShardingSphere-Proxy, and ShardingSphere-Sidecar.
- ShardingSphere-JDBC adopts a non-centralized architecture, which is suitable for high-performance lightweight OLTP (relational database) applications developed by Java;
- ShardingSphere-Proxy provides static interface and heterogeneous language support, and is suitable for OLAP applications (focusing on query and decision-making, generally used in data warehouses) and scenarios for managing and operating separate databases.
A more suitable architecture is a hybrid architecture of JDBC and Proxy:
2、ShardingSphere-JDBC
ShardingSphere-JDBC is a lightweight Java framework that provides additional services in the JDBC layer of Java. It uses the client to directly connect to the database and provides services in the form of jar packages without additional deployment and dependencies. It can be understood as an enhanced version of the JDBC driver, fully compatible with JDBC and various ORM frameworks.
The architecture diagram of ShardingSphere-JDBC is as follows:
Among them, the sharding strategy configuration of ShardingSphere-JDBC is as follows:
- Data source division strategy (sub-database): corresponds to DatabaseShardingStrategy. The target data source used to configure the data to be allocated
- Table sharding strategy (table sharding): Corresponds to TableShardingStrategy. It is used to configure the target table to which the data is allocated. The target table exists in the target data source of the data, so the table division strategy is dependent on the result of the data source division strategy.
Two, ShardingSphere-JDBC quick start
The following mainly introduces the function realization of data fragmentation and read-write separation of ShardingSphere-JDBC.
1. Data fragmentation (sub-database and sub-table)
This article uses 2 databases and 4 tables to simulate the process of sub-database sub-table. First, determine which database to route to based on the userId modulo; and then decide which table to route to based on the modulo of the order id. The model is as follows:
2.1. Introduce the core maven dependency
<dependency>
<groupId>io.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>${sharding-jdbc-version}</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>${mybatis-spring-boot-starter-version}</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid</artifactId>
<version>${druid-version}</version>
</dependency>
2.2. Establish two order libraries, order0 and order1 respectively. There are 2 tables in each library: t_order_0 and t_order_1, a total of 2 libraries and 4 tables to simulate the process of sub-database sub-table. For table-building DML statements, see the path resouces/scripts/t_order.sql in the project project.
2.3. Use mybatis automatic generation tool to generate mapper interface and xml file:
- OrderMapper.xml
- OrderMapper.java
2.4 、 ShardingSphere-JDBC ： （重点 ： ： application-sharding.yml
(1) First configure the data sources: order0, order1, these two data sources correspond to two database names;
(2) Then configure the druid connection pool parameters of the two data sources;
(3) Configure sub-database and sub-table rules: sub-databases are segmented according to user_id. The sub-database rules order$->{user_id % 2}indicate that when user_id% 2 == 0, it will be routed to the order0 database, and when user_id% 2 == 1 it will be routed to the order1 database;
The sharding table is sharded according to order_id. The table sharding rule t_order_$->{order_id % 2}indicates that it will be routed to the t_order_0 table when order_id% 2 == 0, and it will be routed to the t_order_1 table when order_id% 2 == 1;
# Druid连接池参数配置
initialSize: 5 # 初始化大小，最小，最大
minIdle: 5 # 最小连接池数量
maxIdle: 100 # 最大连接池数量
maxActive: 20 # 配置获取连接等待超时的时间
maxWait: 60000 # 检测连接是否有效的sql
timeBetweenEvictionRunsMillis: 60000 # 配置间隔多久才进行一次检测，关闭的空闲连接，单位是毫秒
minEvictableIdleTimeMillis: 300000 # 连接在池中最小生存的时间，单位是毫秒
# sharding-jdbc数据源分片配置
sharding:
jdbc:
# 数据源名称，多数据源以逗号分隔
datasource:
names: order0,order1
# 配置数据源Druid连接池参数
order0:
type: com.alibaba.druid.pool.DruidDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://114.215.176.50:3306/order0?useUnicode=true&characterEncoding=utf8&useSSL=true&serverTimezone=GMT%2B8
username: root
password: *****
initialSize: ${
initialSize}
minIdle: ${
minIdle}
maxActive: ${
maxActive}
maxWait: ${
maxWait}
validationQuery: SELECT 1 FROM DUAL
timeBetweenEvictionRunsMillis: ${
timeBetweenEvictionRunsMillis}
minEvictableIdleTimeMillis: ${
minEvictableIdleTimeMillis}
order1:
type: com.alibaba.druid.pool.DruidDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://114.215.176.50:3306/order1?useUnicode=true&characterEncoding=utf8&useSSL=true&serverTimezone=GMT%2B8
username: root
password: ****
initialSize: ${
initialSize}
minIdle: ${
minIdle}
maxActive: ${
maxActive}
maxWait: ${
maxWait}
validationQuery: SELECT 1 FROM DUAL
timeBetweenEvictionRunsMillis: ${
timeBetweenEvictionRunsMillis}
minEvictableIdleTimeMillis: ${
minEvictableIdleTimeMillis}
# 默认数据源
config:
sharding:
default-data-source-name: order0
# 分库分表配置：根据user_id取模分库，根据order_id取模分表
## 分库配置，按user_id取模运算，奇数在order1库，偶数在order0库
default-database-strategy:
inline:
sharding-column: user_id
algorithm-expression: order$->{
user_id % 2}
## 分表规则配置：按order_id取模运算，奇数在t_order_1表；偶数在t_order_0表
tables:
t_order:
actual-data-nodes: order$->{
0..1}.t_order_$->{
0..1}
table-strategy:
inline:
sharding-column: order_id
algorithm-expression: t_order_$->{
order_id % 2}
2.5 application.yml
spring:
application:
name: sharding-jdbc-demo
profiles:
include: sharding
mybatis:
mapper-locations: classpath:mapper/*.xml
configuration:
log-impl: org.apache.ibatis.logging.stdout.StdOutImpl
2.6. OrderTestCreateUtilTest in the test class uses the test method initOrder provided in the OrderTestCreateUtil class.
When the initOrder() method is executed, it can be found that the orders with userId1 = 111111L are all in the order1 library, and the orders with odd end numbers are in the t_order_0 table, and the rest are in the t_order_1 table.
In the same way, all orders with userId1 = 222222L are in the order1 library;
Of course, the rules for database and table partitioning when querying data are the same, which can be verified by yourself.
2. Read and write separation
ShardingSphere-JDBC currently only supports a single-master and multiple-slave architecture, using a single-master database for adding, updating, and deleting; while queries can use multiple-slave databases, and routing to different databases can be determined by load balancing algorithms.
The read-write separation architecture can improve the read-write performance of the architecture, but at the same time it will bring about the problem of master-slave delay.
The code is omitted.
Three, the principle of ShardingSphere-JDBC sub-database sub-table
The core process of ShardingSphere-JDBC sub-database sub-table is: SQL parsing => executor optimization => SQL routing => SQL rewriting => SQL execution => process composition of result merging.
