05ShardingSphere-JDBC horizontal sharding

1. Prepare the server

As the business expanded, the amount of data in the order table continued to increase, and the database faced storage pressure, so it began to consider horizontal sharding of the order table.

Expand the t_order table to t_order0 and t_order1 in server-order0, and t_order0 and t_order1 in server-order1

Server planning: Usedocker to create the following container

• Server: container nameserver-order0, port3310
• Server: container nameserver-order1, port3311

1.1. Create server-order0 container

step1: Create container:

docker run -d \
-p 3310:3306 \
-v /dongguo/server/order0/conf:/etc/mysql/conf.d \
-v /dongguo/server/order0/data:/var/lib/mysql \
-e MYSQL_ROOT_PASSWORD=123456 \
--name server-order0 \
mysql:8.0.29

step2: Log in to the MySQL server:

#进入容器：
docker exec -it server-order0 env LANG=C.UTF-8 /bin/bash
#进入容器内的mysql命令行
mysql -uroot -p
#修改默认密码插件
ALTER USER 'root'@'%' IDENTIFIED WITH mysql_native_password BY '123456';

step3: Create database:

注意：The ID of horizontal sharding needs to be implemented at the business layer.不能依赖数据库的主键自增

CREATE DATABASE db_order;
USE db_order;
CREATE TABLE t_order0 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) 
);
CREATE TABLE t_order1 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) 
);

1.2. Create server-order1 container

step1: Create container:

docker run -d \
-p 3311:3306 \
-v /dongguo/server/order1/conf:/etc/mysql/conf.d \
-v /dongguo/server/order1/data:/var/lib/mysql \
-e MYSQL_ROOT_PASSWORD=123456 \
--name server-order1 \
mysql:8.0.29

step2: Log in to the MySQL server:

#进入容器：
docker exec -it server-order1 env LANG=C.UTF-8 /bin/bash
#进入容器内的mysql命令行
mysql -uroot -p
#修改默认密码插件
ALTER USER 'root'@'%' IDENTIFIED WITH mysql_native_password BY '123456';

step3: Create database:

Same as server-order0

注意：The ID of horizontal sharding needs to be implemented at the business layer and cannot rely on the primary key of the database to be automatically incremented.

CREATE DATABASE db_order;
USE db_order;
CREATE TABLE t_order0 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) 
);
CREATE TABLE t_order1 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) 
);

View container

View navicat

3310

3311

Later, the server-user service will be needed to start the service.

2. Basic horizontal sharding

2.1. Basic configuration

application.yml

# 应用名称
spring:
  application:
    name: sharding-jdbc-demo
  profiles:
    active: dev
  datasource:
    driver-class-name: org.apache.shardingsphere.driver.ShardingSphereDriver
    #读写分离
#    url: jdbc:shardingsphere:classpath:shardingsphere-readwrite.yaml
    #垂直分片
#    url: jdbc:shardingsphere:classpath:shardingsphere-sharding.yaml
    #水平分片-分库
    url: jdbc:shardingsphere:classpath:shardingsphere-sharding2.yaml
# Swagger配置
swagger:
  # 是否开启swagger
  enabled: true
  # 请求前缀
  pathMapping: /

2.2. Data source configuration

shardingsphere-sharding2.yaml

# 数据源配置
dataSources:
  server-user:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3301/db_user
    username: root
    password: 123456
  server-order0:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3310/db_order
    username: root
    password: 123456
  server-order1:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3311/db_order
    username: root
    password: 123456
#规则配置
rules:
- !SHARDING
  tables:
    # 逻辑表名
    t_user:
      # 值由数据源名 + 表名组成，以小数点分隔。
      actualDataNodes: server-user.t_user
    t_order:
      actualDataNodes: server-order0.t_order_0,server-order0.t_order_1,server-order1.t_order_0,server-order1.t_order_1

#属性配置
props:
  sql-show: true

Modify the primary key strategy of the Order entity class:

Horizontal sharding IDs need to be implemented at the business layer and cannot rely on the primary key of the database to be automatically incremented. Set the ID generation strategy to the snowflake algorithm to generate IDs.

package com.dongguo.shardingjdbc.entity;

import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;

import java.math.BigDecimal;

@TableName("t_order")
@Data
public class Order {
    
    
    //@TableId(type = IdType.AUTO)//依赖数据库的主键自增策略
    @TableId(type = IdType.ASSIGN_ID)//分布式id
    private Long id;
    private String orderNo;
    private Long userId;
    private BigDecimal amount;
}

Before configuring the sharding algorithm, confirm that each shard node is available.

Test: Reserve one shard table node in the above configuration for testing separately, and check whether each shard node is available.

    /**
     * 水平分片：插入数据测试
     */
    @Test
    public void testInsertOrder(){
    
    
        Order order = new Order();
        order.setOrderNo("SP202308140001");
        order.setUserId(1L);
        order.setAmount(new BigDecimal(100));
        orderMapper.insert(order);
    }

shardingsphere-sharding2.yaml

server-order0.t_order0

    t_order:
      actualDataNodes: server-order0.t_order0

2023-08-14 20:49:25.528  INFO 1296 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-14 20:49:25.528  INFO 1296 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1691069489988296705, SP202308140001, 1, 100]

server-order0.t_order1

    t_order:
      actualDataNodes: server-order0.t_order1

2023-08-14 20:50:35.950  INFO 5476 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-14 20:50:35.951  INFO 5476 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1691069785481207809, SP202308140001, 1, 100]

server-order1.t_order0

    t_order:
      actualDataNodes: server-order1.t_order0

2023-08-14 20:51:12.503  INFO 11204 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-14 20:51:12.503  INFO 11204 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1691069939445686274, SP202308140001, 1, 100]

server-order1.t_order1

    t_order:
      actualDataNodes: server-order1.t_order1

2023-08-14 20:51:54.268  INFO 22500 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-14 20:51:54.268  INFO 22500 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1691070114000031746, SP202308140001, 1, 100]

Each shard node is available, and remember to delete the data after each test.

2.3. Line expressions

Optimize the sharded table configuration in the previous step

https://shardingsphere.apache.org/document/current/cn/features/sharding/concept/

Row expressions are used to solve the two main problems of configuration simplification and integration. In the cumbersome configuration of data sharding rules, as the number of data nodes increases, a large number of repeated configurations make the configuration itself difficult to maintain. Row expressions can effectively simplify the data node configuration workload.

For common sharding algorithms, using Java code to implement them does not facilitate unified management of configurations. Writing sharding algorithms through row expressions can effectively store rule configurations together, making it easier to browse and store.

The use of row expression is very intuitive. You only need to use ${ expression } or $->{ expression } to identify the row expression in the configuration. Currently, the configuration of two parts, data node and sharding algorithm, is supported. The content of row expressions uses Groovy's syntax, and row expressions can support all operations that Groovy can support. For example:

${begin..end} represents the range ${[unit1, unit2, unit_x]} represents the enumeration value

${ expression }If there are multiple or $->{ expression } expressions in the line expression, the final result of the entire expression will be based on the value of each sub-expression. The results are Cartesian combined.

shardingsphere-sharding2.yaml

# 数据源配置
dataSources:
  server-user:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3301/db_user
    username: root
    password: 123456
  server-order0:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3310/db_order
    username: root
    password: 123456
  server-order1:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3311/db_order
    username: root
    password: 123456
#规则配置
rules:
- !SHARDING
  tables:
    # 逻辑表名
    t_user:
      # 值由数据源名 + 表名组成，以小数点分隔。
      actualDataNodes: server-user.t_user
    t_order:
      actualDataNodes: server-order${
    
    0..1}.t_order${
    
    0..1}

#属性配置
props:
  sql-show: true

2.4. Sharding algorithm configuration

ShardingSphere provides a variety of built-in sharding algorithms, which can be divided into automatic sharding algorithms, standard sharding algorithms, compound sharding algorithms and Hint sharding algorithms according to type, which can meet the needs of most users' business scenarios. In addition, considering the complexity of business scenarios, the built-in algorithm also provides a way to customize the sharding algorithm, and users can complete complex sharding logic by writing Java code. It should be noted that the sharding logic of the automatic sharding algorithm is automatically managed by ShardingSphere and needs to be used by configuring autoTables sharding rules.

Horizontal sub-library

Fragmentation rules: Whenuser_id is an even number in the order table, data is insertedserver-order0服务器, and when user_id is an odd number, Data insertserver-order1服务器. The advantage of this sharding is that the order data of the same user will be inserted into the same server, making it more efficient to query a user's order.

Sharing algorithm related references:

https://shardingsphere.apache.org/document/current/cn/user-manual/common-config/builtin-algorithm/sharding/

shardingsphere-sharding2.yaml

Standard Sharding Algorithm - Row Expression Sharding Algorithm

# 数据源配置
dataSources:
  server-user:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3301/db_user
    username: root
    password: 123456
  server-order0:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3310/db_order
    username: root
    password: 123456
  server-order1:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3311/db_order
    username: root
    password: 123456
#规则配置
rules:
- !SHARDING
  tables:
    # 逻辑表名
    t_user:
      # 值由数据源名 + 表名组成，以小数点分隔。
      actualDataNodes: server-user.t_user
    t_order:
      actualDataNodes: server-order${
    
    0..1}.t_order${
    
    0..1}
      #分库策略
      databaseStrategy:
        standard:
          # 分片列名称
          shardingColumn: user_id
          # 分片算法名称
          shardingAlgorithmName: alg_inline_userid
  # 分片算法配置
  shardingAlgorithms:
    # 标准分片算法-行表达式分片算法
    alg_inline_userid:
      #基于行表达式的分片算法
      type: INLINE
      props:
        algorithm-expression: server-order${
    
    user_id % 2}
#属性配置
props:
  sql-show: true

The sharding algorithm alg_inline_userid is configured

Note: In some previous versions, the value corresponding to shardingAlgorithmName did not seem to support underscores. In ShardingSphere 5.4.0 version, underscore naming is supported.

If the version does not support underscores, the error message is as follows:

Caused by: org.apache.shardingsphere.spi.exception.ServiceProviderNotFoundException: No implementation class load from SPI `org.apache.shardingsphere.sharding.spi.ShardingAlgorithm` with type `null`.

In order to facilitate testing, first set up the test only on the t_order0 table to test the database sharding strategy - row expression sharding algorithm (currently no table sharding strategy is configured)

shardingsphere-sharding2.yaml

#规则配置
rules:
- !SHARDING
  tables:
    # 逻辑表名
    t_user:
      # 值由数据源名 + 表名组成，以小数点分隔。
      actualDataNodes: server-user.t_user
    t_order:
      actualDataNodes: server-order${
    
    0..1}.t_order0
      #分库策略
      databaseStrategy:
        standard:
          # 分片列名称
          shardingColumn: user_id
          # 分片算法名称
          shardingAlgorithmName: alg_inline_userid
  # 分片算法配置
  shardingAlgorithms:
    # 标准分片算法-行表达式分片算法
    alg_inline_userid:
      type: INLINE
      props:
        algorithm-expression: server-order${
    
    user_id % 2}

    /**
     * 水平分片：分库插入数据测试
     */
    @Test
    public void testInsertOrderDatabaseStrategy(){
    
    

        for (long i = 1; i <= 4; i++) {
    
    
            Order order = new Order();
            order.setOrderNo("SP20230814000" + i);
            order.setUserId(i);
            order.setAmount(new BigDecimal(100));
            orderMapper.insert(order);
        }
    }

2023-08-17 18:58:27.716  INFO 8660 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 18:58:27.716  INFO 8660 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692128729070862338, SP202308140001, 1, 100]
2023-08-17 18:58:27.742  INFO 8660 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 18:58:27.743  INFO 8660 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692128753561403393, SP202308140002, 2, 100]
2023-08-17 18:58:27.749  INFO 8660 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 18:58:27.749  INFO 8660 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692128753561403394, SP202308140003, 3, 100]
2023-08-17 18:58:27.754  INFO 8660 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 18:58:27.754  INFO 8660 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692128753561403395, SP202308140004, 4, 100]

The data with user_id 1,3 (odd numbers) is saved in the t_order_0 table of the server-order1 data source, and the data with user_id 2,4 (even numbers) is saved in the t_order_0 table of the server-order0 data source.

server-order0

server-order1

Level score table

Fragmentation rules: In the server-order0 server, when id为偶数时，数据插入t_order0，id is an odd number, the data is inserted into t_order1.

shardingsphere-sharding3.yaml

# 数据源配置
dataSources:
  server-user:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3301/db_user
    username: root
    password: 123456
  server-order0:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3310/db_order
    username: root
    password: 123456
  server-order1:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3311/db_order
    username: root
    password: 123456
#规则配置
rules:
- !SHARDING
  tables:
    # 逻辑表名
    t_order:
      actualDataNodes: server-order0.t_order${
    
    0..1}
      #分表策略
      tableStrategy:
        standard:
          # 分片列名称
          shardingColumn: id
          # 分片算法名称
          shardingAlgorithmName: alg_inline_id
  # 分片算法配置
  shardingAlgorithms:
    # 标准分片算法-行表达式分片算法
    alg_inline_id:
      #基于行表达式的分片算法
      type: INLINE
      props:
        algorithm-expression: t_order${
    
    id % 2} 
#属性配置
props:
  sql-show: true

implement

    /**
     * 水平分片：分表插入数据测试
     */
    @Test
    public void testInsertOrderTableStrategy(){
    
    
        for (long i = 1; i <= 10; i++) {
    
    
            Order order = new Order();
            order.setId(i);
            order.setOrderNo("SP20230814000" + i);
            order.setUserId(i);
            order.setAmount(new BigDecimal(100));
            orderMapper.insert(order);
        }
    }

The IDs 2 and 4 are inserted into the t_order0 table, and the IDs 1 and 3 are inserted into the t_order1 table.

2023-08-17 21:59:30.107  INFO 24352 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 21:59:30.107  INFO 24352 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1, SP202308140001, 1, 100]
2023-08-17 21:59:30.139  INFO 24352 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 21:59:30.139  INFO 24352 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [2, SP202308140002, 2, 100]
2023-08-17 21:59:30.144  INFO 24352 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 21:59:30.144  INFO 24352 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [3, SP202308140003, 3, 100]
2023-08-17 21:59:30.148  INFO 24352 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 21:59:30.148  INFO 24352 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [4, SP202308140004, 4, 100]
2023-08-17 21:59:30.151  INFO 24352 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )

Sub-database and sub-table

Fragmentation rules: Whenuser_id is an even number in the order table, data is insertedserver-order0服务器, and when user_id is an odd number, Data insertserver-order1服务器.

When id为偶数时，数据插入t_order0，id is an odd number, the data is inserted into t_order1.

shardingsphere-sharding4.yaml

# 数据源配置
dataSources:
  server-user:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3301/db_user
    username: root
    password: 123456
  server-order0:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3310/db_order
    username: root
    password: 123456
  server-order1:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3311/db_order
    username: root
    password: 123456
#规则配置
rules:
- !SHARDING
  tables:
    # 逻辑表名
    t_order:
      actualDataNodes: server-order${
    
    0..1}.t_order${
    
    0..1}
      #分库策略
      databaseStrategy:
        standard:
          # 分片列名称
          shardingColumn: user_id
          # 分片算法名称
          shardingAlgorithmName: alg_inline_userid
      #分表策略
      tableStrategy:
        standard:
          # 分片列名称
          shardingColumn: id
          # 分片算法名称
          shardingAlgorithmName: alg_inline_id
  # 分片算法配置
  shardingAlgorithms:
    # 标准分片算法-行表达式分片算法
    alg_inline_userid:
      #基于行表达式的分片算法
      type: INLINE
      props:
        algorithm-expression: server-order${
    
    user_id % 2}
    alg_inline_id:
      #基于行表达式的分片算法
      type: INLINE
      props:
        algorithm-expression: t_order${
    
    id % 2}
#属性配置
props:
  sql-show: true

    /**
     * 水平分片：分库分表插入数据
     */
    @Test
    public void testInsertOrderDatabaseAndTableStrategy(){
    
    
        for (long i = 1; i < 5; i++) {
    
    

            Order order = new Order();
            order.setId(i);
            order.setOrderNo("SP20230814000" + i);
            order.setUserId(1L);
            order.setAmount(new BigDecimal(100));
            orderMapper.insert(order);
        }

        for (long i = 5; i < 9; i++) {
    
    
            Order order = new Order();
            order.setId(i);
            order.setOrderNo("SP20230814000" + i);
            order.setUserId(2L);
            order.setAmount(new BigDecimal(100));
            orderMapper.insert(order);
        }
    }

You can see that the userId with an even number is inserted into server-order0, and the userId with an odd number is inserted into server-order1;

The even-numbered id is inserted into t_table0, and the odd-numbered id is inserted into table1.

2023-08-17 22:09:59.399  INFO 23180 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 22:09:59.399  INFO 23180 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1, SP202308140001, 1, 100]
2023-08-17 22:09:59.443  INFO 23180 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 22:09:59.443  INFO 23180 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [2, SP202308140002, 1, 100]
2023-08-17 22:09:59.448  INFO 23180 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 22:09:59.448  INFO 23180 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [3, SP202308140003, 1, 100]
2023-08-17 22:09:59.452  INFO 23180 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 22:09:59.452  INFO 23180 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [4, SP202308140004, 1, 100]
2023-08-17 22:09:59.457  INFO 23180 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 22:09:59.457  INFO 23180 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [5, SP202308140005, 2, 100]
2023-08-17 22:09:59.462  INFO 23180 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 22:09:59.462  INFO 23180 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [6, SP202308140006, 2, 100]
2023-08-17 22:09:59.465  INFO 23180 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 22:09:59.465  INFO 23180 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [7, SP202308140007, 2, 100]
2023-08-17 22:09:59.468  INFO 23180 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-17 22:09:59.468  INFO 23180 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [8, SP202308140008, 2, 100]

Automatic sharding algorithm

In order to help users better use the sharding function, reduce configuration complexity and improve user experience, Apache ShardingSphere 5.0.0 version launches a new sharding configuration method: AutoTable.

The sharding logic of the automatic sharding algorithm is automatically managed by ShardingSphere and needs to be used by configuring autoTables sharding rules.

Modulo sharding algorithm

shardingsphere-sharding5.yaml

Using the modulo sharding algorithm, a total of 4 shards, two data sources, and two shards in each data source

# 数据源配置
dataSources:
  server-order0:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3310/db_order
    username: root
    password: 123456
  server-order1:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3311/db_order
    username: root
    password: 123456
#规则配置
rules:
  - !SHARDING
    autoTables:
      # 逻辑表名
      t_order:
        actualDataSources : server-order${
    
    0..1}
        #分片策略
        shardingStrategy:
          standard:
            # 分片列名称
            shardingColumn: user_id
            # 分片算法名称
            shardingAlgorithmName: alg_mod
    # 分片算法配置
    shardingAlgorithms:
      alg_mod:
        #自动分片算法-取模分片算法
        type: MOD
        props:
          sharding-count: 4
#属性配置
props:
  sql-show: true

One thing worth noting here is that the sharding logic of the automatic sharding algorithm is automatically managed by ShardingSphere, so the tables created before may not comply with the automatic sharding rules. Shardingsphere can create corresponding tables for us based on the automatic sharding rules.

    /**
     * 自动分片创建数据库表
     */
    @Autowired
    JdbcTemplate jdbcTemplate;

    @Test
    public void testCreateAutoOrderMod() {
    
    
        jdbcTemplate.execute("CREATE TABLE t_order (\n" +
                "  id BIGINT,\n" +
                "  order_no VARCHAR(30),\n" +
                "  user_id BIGINT,\n" +
                "  amount DECIMAL(10,2),\n" +
                "  PRIMARY KEY(id) USING BTREE\n" +
                ") ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;");
    }

Execute after deleting the previously created table

2023-08-18 07:59:52.896  INFO 10544 --- [           main] ShardingSphere-SQL                       : Logic SQL: CREATE TABLE t_order (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;
2023-08-18 07:59:52.896  INFO 10544 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: CREATE TABLE t_order_1 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;
2023-08-18 07:59:52.896  INFO 10544 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: CREATE TABLE t_order_3 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;
2023-08-18 07:59:52.896  INFO 10544 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: CREATE TABLE t_order_0 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;
2023-08-18 07:59:52.896  INFO 10544 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: CREATE TABLE t_order_2 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;

It is obvious that ShardingSphere created t_order_0 and t_order_2 for us in server-order0 according to the sharding rules; it created t_order_1 and t_order_3 in server-order1, which are somewhat different from the table names we created manually before.

Insert data

    /**
     * 分片插入数据测试
     */
    @Test
    public void testInsertOrderShardingStrategy(){
    
    

        for (long i = 1; i < 5; i++) {
    
    
            Order order = new Order();
            order.setOrderNo("SP20230814000" + i);
            order.setUserId(1L);
            order.setAmount(new BigDecimal(100));
            orderMapper.insert(order);
        }

        for (long i = 5; i < 9; i++) {
    
    
            Order order = new Order();
            order.setOrderNo("SP20230814000" + i);
            order.setUserId(2L);
            order.setAmount(new BigDecimal(100));
            orderMapper.insert(order);
        }

        for (long i = 10; i < 15; i++) {
    
    
            Order order = new Order();
            order.setOrderNo("SP20230814000" + i);
            order.setUserId(3L);
            order.setAmount(new BigDecimal(100));
            orderMapper.insert(order);
        }
        for (long i = 15; i < 19; i++) {
    
    
            Order order = new Order();
            order.setOrderNo("SP20230814000" + i);
            order.setUserId(4L);
            order.setAmount(new BigDecimal(100));
            orderMapper.insert(order);
        }
    }

According to the modulo sharding algorithm, the data with userId 1 will be inserted into the t_order1 table of the first shard server-order1.

The data with userId 2 will be inserted into the t_order2 table of the first shard server-order0

The data with userId 3 will be inserted into the t_order3 table of the first shard server-order1

The data with userId 4 will be inserted into the t_order0 table of the first shard server-order1

Execution is like this

2023-08-18 08:06:31.169  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.169  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327051316428801, SP202308140001, 1, 100]
2023-08-18 08:06:31.207  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.208  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074754199554, SP202308140002, 1, 100]
2023-08-18 08:06:31.212  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.212  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074821308417, SP202308140003, 1, 100]
2023-08-18 08:06:31.214  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.214  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074821308418, SP202308140004, 1, 100]
2023-08-18 08:06:31.215  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.215  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_2  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074821308419, SP202308140005, 2, 100]
2023-08-18 08:06:31.219  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.219  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_2  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074821308420, SP202308140006, 2, 100]
2023-08-18 08:06:31.223  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.223  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_2  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074821308421, SP202308140007, 2, 100]
2023-08-18 08:06:31.225  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.225  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_2  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074821308422, SP202308140008, 2, 100]
2023-08-18 08:06:31.227  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.227  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_3  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074888417282, SP2023081400010, 3, 100]
2023-08-18 08:06:31.229  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.229  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_3  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074888417283, SP2023081400011, 3, 100]
2023-08-18 08:06:31.231  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.231  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_3  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074888417284, SP2023081400012, 3, 100]
2023-08-18 08:06:31.233  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.233  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_3  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074888417285, SP2023081400013, 3, 100]
2023-08-18 08:06:31.235  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.235  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_3  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074888417286, SP2023081400014, 3, 100]
2023-08-18 08:06:31.236  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.236  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074888417287, SP2023081400015, 4, 100]
2023-08-18 08:06:31.239  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.239  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074888417288, SP2023081400016, 4, 100]
2023-08-18 08:06:31.241  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.241  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074888417289, SP2023081400017, 4, 100]
2023-08-18 08:06:31.243  INFO 21164 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-18 08:06:31.244  INFO 21164 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692327074951331842, SP2023081400018, 4, 100]

Hash modulo sharding algorithm

shardingsphere-sharding6.yaml

# 数据源配置
dataSources:
  server-user:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3301/db_user
    username: root
    password: 123456
  server-order0:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3310/db_order
    username: root
    password: 123456
  server-order1:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.122.150:3311/db_order
    username: root
    password: 123456
#规则配置
rules:
- !SHARDING
  autoTables:
    # 逻辑表名
    t_order:
      actualDataSources : server-order${
    
    0..1}
      #分片策略
      shardingStrategy:
        standard:
          # 分片列名称
          shardingColumn: order_no
          # 分片算法名称
          shardingAlgorithmName: alg_hash_mod
  # 分片算法配置
  shardingAlgorithms:
    alg_hash_mod:
      type: HASH_MOD
      props:
        sharding-count: 4
#属性配置
props:
  sql-show: true

order_no is a string type, with a value such as "SP2023081400001". It cannot be modulo directly, so the modulo sharding algorithm cannot be used for order_no, so the hash modulo sharding algorithm is used for order_no. Note that the sharding rules here are The hash value of order_no. If the hash value is modulo 4, you can see which shard the order falls on.

implement

    /**
     * 哈希取模
     */
    @Test
    public void testInsertOrderHashModStrategy(){
    
    

        for (long i = 1; i < 10; i++) {
    
    
            Order order = new Order();
            order.setOrderNo("SP20230814000" + i);
            order.setUserId(1L);
            order.setAmount(new BigDecimal(100));
            orderMapper.insert(order);
            System.out.println("对订单号的hash值取模：" + order.getOrderNo().hashCode() % 4);
        }
    }

2023-08-19 08:24:05.112  INFO 20980 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-19 08:24:05.112  INFO 20980 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692693855566049282, SP202308140001, 1, 100]
对订单号的hash值取模：0
2023-08-19 08:24:05.164  INFO 20980 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-19 08:24:05.164  INFO 20980 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_3  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692693883210706946, SP202308140002, 1, 100]
对订单号的hash值取模：-3
2023-08-19 08:24:05.176  INFO 20980 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-19 08:24:05.176  INFO 20980 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_2  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692693883277815809, SP202308140003, 1, 100]
对订单号的hash值取模：-2
2023-08-19 08:24:05.182  INFO 20980 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-19 08:24:05.182  INFO 20980 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692693883340730370, SP202308140004, 1, 100]
对订单号的hash值取模：-1
2023-08-19 08:24:05.186  INFO 20980 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-19 08:24:05.186  INFO 20980 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692693883340730371, SP202308140005, 1, 100]
对订单号的hash值取模：0
2023-08-19 08:24:05.191  INFO 20980 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-19 08:24:05.191  INFO 20980 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_3  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692693883340730372, SP202308140006, 1, 100]
对订单号的hash值取模：-3
2023-08-19 08:24:05.196  INFO 20980 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-19 08:24:05.196  INFO 20980 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_2  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692693883407839233, SP202308140007, 1, 100]
对订单号的hash值取模：-2
2023-08-19 08:24:05.200  INFO 20980 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-19 08:24:05.200  INFO 20980 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692693883407839234, SP202308140008, 1, 100]
对订单号的hash值取模：-1
2023-08-19 08:24:05.204  INFO 20980 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-19 08:24:05.204  INFO 20980 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1692693883407839235, SP202308140009, 1, 100]
对订单号的hash值取模：0

Here is just a brief introduction to AuTotable. For more sharding algorithms, please refer to:

https://blog.csdn.net/ctwy291314/article/details/130561221

2.5. Distributed sequence algorithm

Realization motivation

In traditional database software development, automatic primary key generation technology is a basic requirement. Each database also provides corresponding support for this requirement, such as MySQL's auto-increment key, Oracle's auto-increment sequence, etc. After data is sharded, generating globally unique primary keys from different data nodes is a very difficult problem. The auto-increment keys between different actual tables in the same logical table cannot sense each other, resulting in duplicate primary keys. Although collisions can be avoided by constraining the initial value and step size of the auto-increasing primary key, additional operation and maintenance rules need to be introduced, making the solution lack completeness and scalability.

There are currently many third-party solutions that can perfectly solve this problem, such as UUID, etc. that rely on specific algorithms to self-generate unique keys, or by introducing primary key generation services. In order to facilitate users and meet the needs of different users in different usage scenarios, Apache ShardingSphere not only provides built-in distributed primary key generators, such as UUID and SNOWFLAKE, but also extracts the interface of the distributed primary key generator to facilitate users to implement their own customization The auto-incrementing primary key generator.

Built-in primary key generator

UUID

uses UUID.randomUUID() to generate distributed primary keys.

SNOWFLAKE

The primary key generation strategy for each table can be configured in the sharding rule configuration module. By default, the snowflake algorithm is used to generate 64-bit long integer data.

Snowflake algorithm is a distributed primary key generation algorithm announced by Twitter. It can ensure the non-repetition of primary keys of different processes and the orderliness of primary keys of the same process.

Implementation principle

In the same process, it first ensures non-duplication through the time bit, and if the times are the same, it guarantees it through the sequence bit. At the same time, because the time bits are monotonically increasing, and if each server is generally time synchronized, the generated primary keys can be considered generally ordered in the distributed environment, which ensures the efficiency of inserting index fields. For example, the primary key of MySQL's Innodb storage engine.

The primary key generated using the snowflake algorithm, the binary representation contains 4 parts, from high to low: 1-bit sign bit, 41-bit timestamp bit, 10-bit working process bit and 12-bit sequence number bit.

• Sign bit (1bit)

The reserved sign bit is always zero.

• Timestamp bit (41bit)

The number of milliseconds that a 41-bit timestamp can hold is 2 to the 41st power, and the number of milliseconds used in a year is:365 * 24 * 60 * 60 * 1000. It can be seen through calculation:

Math.pow(2, 41) / (365 * 24 * 60 * 60 * 1000L);

The result is approximately 69.73 years. The time epoch of Apache ShardingSphere's snowflake algorithm starts from 2016年11月1日 zero and can be used until 2086. It is believed to be able to meet the requirements of most systems.

• Worker process bit (10bit)

This flag is unique within the Java process. If it is a distributed application deployment, it should be ensured that the ID of each worker process is different. This value defaults to 0 and can be set via properties.

• Serial number bit (12bit)

This sequence is used to generate different IDs within the same millisecond. If the number generated in this millisecond exceeds 4096 (2 to the power of 12), the generator will wait until the next millisecond to continue generating.

The detailed structure of the snowflake algorithm primary key is shown in the figure below.

Clock set back

Server clock rollback can result in duplicate sequences, so the default distributed primary key generator provides a maximum tolerated clock rollback in milliseconds. If the clock dialback time exceeds the maximum tolerated millisecond threshold, the program will report an error; if it is within the tolerable range, the default distributed primary key generator will wait for the clock to be synchronized to the time of the last primary key generation before continuing to work. The default value for the maximum tolerated clock rollback in milliseconds is 0 and can be set via properties.

Snowflake algorithm:

Horizontal sharding requires attention to the global sequence, because you cannot simply use database-based primary key auto-increment.

There are two solutions here: one is based on the id strategy of MyBatisPlus; the other is the global sequence configuration of ShardingSphere-JDBC.

基于MyBatisPlus的id策略：Set the id of the Order class to the following form

@TableId(type = IdType.ASSIGN_ID)
private Long id;

基于ShardingSphere-JDBC的全局序列配置: Choose one from the previous MyBatisPlus strategy

Configure distributed sequence policy configuration and distributed sequence algorithm configuration

#规则配置
rules:
- !SHARDING
  autoTables:
    # 逻辑表名
    t_order:
      actualDataSources : server-order${
    
    0..1}
      #分片策略
      shardingStrategy:
        standard:
          # 分片列名称
          shardingColumn: order_no
          # 分片算法名称
          shardingAlgorithmName: alg_hash_mod
      #分布式序列策略配置
      keyGenerateStrategy:
      # 分布式序列列名称
        column: id
        # 分布式序列算法名称
        keyGeneratorName: snowflake
  # 分片算法配置
  shardingAlgorithms:
    alg_hash_mod:
      type: HASH_MOD
      props:
        sharding-count: 4
  # 分布式序列算法配置      
  keyGenerator
    snowflake:
      # 分布式序列算法类型
      type: SNOWFLAKE      
#属性配置
props:
  sql-show: true

At this time, you need to modify the id policy in the entity class to the following form:

//当配置了shardingsphere-jdbc的分布式序列时，自动使用shardingsphere-jdbc的分布式序列
//当没有配置shardingsphere-jdbc的分布式序列时，自动依赖数据库的主键自增策略
@TableId(type = IdType.AUTO)

Execution method

    /**
     * 哈希取模
     */
    @Test
    public void testInsertOrderHashModStrategy(){
    
    

        for (long i = 1; i < 10; i++) {
    
    
            Order order = new Order();
            order.setOrderNo("SP20230814000" + i);
            order.setUserId(1L);
            order.setAmount(new BigDecimal(100));
            orderMapper.insert(order);
            System.out.println("对订单号的hash值取模：" + order.getOrderNo().hashCode() % 4);
        }
    }
}

2023-08-20 10:12:23.145  INFO 27620 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-20 10:12:23.145  INFO 27620 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1693083499130646529, SP202308140001, 1, 100]
对订单号的hash值取模：0
2023-08-20 10:12:23.179  INFO 27620 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-20 10:12:23.179  INFO 27620 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_3  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1693083525789642754, SP202308140002, 1, 100]
对订单号的hash值取模：-3
2023-08-20 10:12:23.185  INFO 27620 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-20 10:12:23.185  INFO 27620 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_2  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1693083525789642755, SP202308140003, 1, 100]
对订单号的hash值取模：-2
2023-08-20 10:12:23.188  INFO 27620 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-20 10:12:23.188  INFO 27620 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1693083525789642756, SP202308140004, 1, 100]
对订单号的hash值取模：-1
2023-08-20 10:12:23.192  INFO 27620 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-20 10:12:23.192  INFO 27620 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1693083525852557313, SP202308140005, 1, 100]
对订单号的hash值取模：0
2023-08-20 10:12:23.196  INFO 27620 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-20 10:12:23.196  INFO 27620 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_3  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1693083525852557314, SP202308140006, 1, 100]
对订单号的hash值取模：-3
2023-08-20 10:12:23.199  INFO 27620 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-20 10:12:23.200  INFO 27620 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_2  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1693083525852557315, SP202308140007, 1, 100]
对订单号的hash值取模：-2
2023-08-20 10:12:23.203  INFO 27620 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-20 10:12:23.203  INFO 27620 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: INSERT INTO t_order_1  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1693083525852557316, SP202308140008, 1, 100]
对订单号的hash值取模：-1
2023-08-20 10:12:23.206  INFO 27620 --- [           main] ShardingSphere-SQL                       : Logic SQL: INSERT INTO t_order  ( id,
order_no,
user_id,
amount )  VALUES  ( ?,
?,
?,
? )
2023-08-20 10:12:23.207  INFO 27620 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: INSERT INTO t_order_0  ( id,
order_no,
user_id,
amount )  VALUES  (?, ?, ?, ?) ::: [1693083525852557317, SP202308140009, 1, 100]
对订单号的hash值取模：0

Distributed Sequence Algorithm

https://shardingsphere.apache.org/document/current/cn/user-manual/common-config/builtin-algorithm/keygen/