SpringBoot2.1.0 + Sharding-JDBC4.0.0实现数据分片Demo

目录

建库建表

创建工程

pom.xml

项目文件结构

主要分片策略

功能实现

实体类

Mapper

Service

Hint实现类

测试类

总结

---

Sharding-JDBC官方文档：https://shardingsphere.apache.org/document/current/cn/overview/

本文参考官方文档，基于SpringBoot 2.1.0.RELEASE，与Sharding-JDBC 4.0.0-RC1版本，写了一个demo，实现了数据分片的基本功能。

建库建表

这里在同一台MySQL服务器上建立了两个数据库实例：ds0、ds1

然后在ds0上创建：t_order0、t_order_item0、t_config

然后在ds1上创建：t_order1、t_order_item1、t_config

-- 创建数据源
CREATE SCHEMA IF NOT EXISTS ds0;
CREATE SCHEMA IF NOT EXISTS ds1;
-- ds0建表
DROP TABLE IF EXISTS ds0.t_order0;
CREATE TABLE ds0.t_order0 (
	order_id INT PRIMARY KEY, 
	user_id INT NOT NULL, 
	config_id INT NOT NULL, 
	remark VARCHAR(50),
	create_time TIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
	last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
DROP TABLE IF EXISTS ds0.t_order1;
CREATE TABLE ds0.t_order1 (
	order_id INT PRIMARY KEY, 
	user_id INT NOT NULL, 
	config_id INT NOT NULL, 
	remark VARCHAR(50),
	create_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

DROP TABLE IF EXISTS ds0.t_order_item0;
CREATE TABLE ds0.t_order_item0 (
	item_id BIGINT PRIMARY KEY AUTO_INCREMENT,
	order_id BIGINT NOT NULL,
	remark VARCHAR(50),
	create_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
DROP TABLE IF EXISTS ds0.t_order_item1;
CREATE TABLE ds0.t_order_item1 (
	item_id BIGINT PRIMARY KEY AUTO_INCREMENT,
	order_id BIGINT NOT NULL,
	remark VARCHAR(50),
	create_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
-- 广播表
DROP TABLE IF EXISTS ds0.t_config;
CREATE TABLE IF NOT EXISTS ds0.t_config (
	id INT PRIMARY KEY, 
	remark VARCHAR(50),
	create_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

-- ds1建表
DROP TABLE IF EXISTS ds1.t_order0;
CREATE TABLE ds1.t_order0 (
	order_id INT PRIMARY KEY, 
	user_id INT NOT NULL, 
	config_id INT NOT NULL, 
	remark VARCHAR(50),
	create_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
DROP TABLE IF EXISTS ds1.t_order1;
CREATE TABLE ds1.t_order1 (
	order_id INT PRIMARY KEY, 
	user_id INT NOT NULL, 
	config_id INT NOT NULL, 
	remark VARCHAR(50),
	create_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

DROP TABLE IF EXISTS ds1.t_order_item0;
CREATE TABLE ds1.t_order_item0 (
	item_id BIGINT PRIMARY KEY AUTO_INCREMENT,
	order_id BIGINT NOT NULL,
	remark VARCHAR(50),
	create_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
DROP TABLE IF EXISTS ds1.t_order_item1;
CREATE TABLE ds1.t_order_item1 (
	item_id BIGINT PRIMARY KEY AUTO_INCREMENT,
	order_id BIGINT NOT NULL,
	remark VARCHAR(50),
	create_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
-- 广播表
DROP TABLE IF EXISTS ds1.t_config;
CREATE TABLE ds1.t_config (
	id INT PRIMARY KEY, 
	remark VARCHAR(50),
	create_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
	last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

创建工程

创建Maven工程，这里SpringBoot使用的是2.1.0.RELEASE版本（为了与实际工作中的项目一致），ORM使用的MyBatis，

Sharding-JDBC使用的是当前最新版本4.0.0-RC1。

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.1.0.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.hujy</groupId>
	<artifactId>sharding-jdbc-demo</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>sharding-jdbc-demo</name>
	<description>Demo project for Spring Boot</description>

	<properties>
		<java.version>1.8</java.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>

		<dependency>
			<groupId>org.mybatis.spring.boot</groupId>
			<artifactId>mybatis-spring-boot-starter</artifactId>
			<version>2.1.0</version>
		</dependency>

		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
			<scope>runtime</scope>
		</dependency>

		<dependency>
			<groupId>org.projectlombok</groupId>
			<artifactId>lombok</artifactId>
			<optional>true</optional>
		</dependency>

		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-lang3</artifactId>
		</dependency>

		<dependency>
			<groupId>com.alibaba</groupId>
			<artifactId>druid</artifactId>
			<version>1.1.10</version>
		</dependency>

		<dependency>
			<groupId>org.apache.shardingsphere</groupId>
			<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
			<version>4.0.0-RC1</version>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>
</project>

项目文件结构

主要分片策略

前面在多个数据源中共创建了三类表：

分片表主表：t_order$，主要字段：order_id、user_id、config_id

分片表子表：t_order_item$，主要字段：item_id、order_id、config_id

配置表：t_config，主要字段：id

大体分片思路为：先通过user_id进行分库，再根据order_id进行分表，达到用户维度的数据在同一个数据库中，订单维度的数据在同一套表中。

具体配置如下：

application-sharding.yml

spring:
  shardingsphere:
    datasource:
      names: ds0,ds1
      ds0:
        type: com.zaxxer.hikari.HikariDataSource
        driver-class-name: com.mysql.cj.jdbc.Driver
        jdbc-url: jdbc:mysql://192.168.115.133:3306/ds0
        username: hjy
        password: 123456
      ds1:
        type: com.zaxxer.hikari.HikariDataSource
        driver-class-name: com.mysql.cj.jdbc.Driver
        jdbc-url: jdbc:mysql://192.168.115.133:3306/ds1
        username: hjy
        password: 123456
    sharding:
      tables:
        t_order:
          actual-data-nodes: ds$->{0..1}.t_order$->{0..1}
          ## 指定分库规则
          database-strategy:
            inline:
              sharding-column: user_id
              algorithm-expression: ds$->{user_id % 2}
          ## 指定分表规则
          table-strategy:
            inline:
              sharding-column: order_id
              algorithm-expression: t_order$->{order_id % 2}

        t_order_item:
          actual-data-nodes: ds$->{0..1}.t_order_item$->{0..1}
          ## 通过hint方式自定义分库规则
          database-strategy:
            hint:
              algorithmClassName: com.hujy.demo.hint.HintSharding
          ## 指定分表规则
          table-strategy:
            inline:
              sharding-column: order_id
          ## 生成分布式主键
          key-generator:
            column: item_id
            type: SNOWFLAKE

      ## 绑定主表与子表，避免关联查询导致的全数据源路由
      binding-tables: t_order,t_order_item

      ## 配置广播表：以广播的形式保存（如果只涉及查询的话可以不配置，会随机取一个数据源）
      broadcast-tables: t_config

    ## 打印sql
    props:
      sql:
        show: true

其中分库策略为通过user_id对2取模，2指的是数据库分片数，由于t_order_item$表中，没有user_id字段，所以无法使用SQL的行表达式策略，需要使用hint方式实现通过user_id的强制路由。

分表的逻辑比较简单，因为两张分片表都有order_id字段，所有配置中指定的table-strategy相同，通过order_id对2取模，2指的是表分片数。

分片的主表与子表还需要通过binding-tables属性指定绑定关系，避免在做关联查询时导致的全数据源路由。

t_config表为配置表，它在每个数据源中保存的内容都是一样的，所以在对其进行写操作时，希望以广播的形式保存。所以通过broadcast-tables属性将其指定为广播表。

功能实现

实体类

Config

@Setter
@Getter
@ToString
public class Config {

    private Integer id;

    private String remark;

    private Date createTime;

    private Date lastModifyTime;
}

Order

@Getter
@Setter
@ToString
public class Order {

    private Integer orderId;

    private Integer userId;

    private Integer configId;

    private String remark;

    private Date createTime;

    private Date lastModifyTime;
}

OrderItem

@Getter
@Setter
@ToString
public class OrderItem {
    
    private Long itemId;

    private Integer orderId;

    private String remark;

    private Date createTime;

    private Date lastModifyTime;
}

Mapper

ConfigMapper

@Mapper
public interface ConfigMapper {
    @Insert("insert into t_config(id,remark) values(#{id},#{remark})")
    Integer save(Config config);

    @Select("select * from t_config  where id = #{id}")
    Config selectById(Integer id);
}

OrderMapper

@Mapper
public interface OrderMapper {

    @Insert("insert into t_order(order_id,user_id,config_id,remark) values(#{orderId},#{userId},#{configId},#{remark})")
    Integer save(Order order);

    @Select("select order_id orderId, user_id userId, config_id configId, remark from t_order  where user_id = #{userId}")
    Order selectByUserId(Integer userId);

    @Select("select o.order_id orderId, o.user_id userId, o.config_id configId, o.remark from " +
            "t_order o inner join t_order_item i on o.order_id = i.order_id " +
            "where o.user_id =#{userId} and o.order_id =#{orderId}")
    List<Order> selectOrderJoinOrderItem(Integer userId, Integer orderId);

    @Select("select  o.order_id orderId, o.user_id userId, o.config_id configId, o.remark " +
            "from t_order o inner join t_config c on o.config_id = c.id " +
            "where o.user_id =#{userId} and o.order_id =#{orderId}")
    List<Order> selectOrderJoinConfig(Integer userId, Integer orderId);
}

OrderItemMapper

@Mapper
public interface OrderItemMapper {
    @Insert("insert into t_order_item(order_id,remark) values(#{orderId},#{remark})")
    Integer save(OrderItem orderItem);
}

Service

OrderService

public interface OrderService {
    Integer saveOrder(Order order);

    Integer saveOrderItem(OrderItem orderItem, Integer userId);

    Order selectByUserId(Integer userId);

    List<Order> selectOrderJoinOrderItem(Integer userId, Integer orderId);

    List<Order> selectOrderJoinOrderItemNoSharding(Integer userId, Integer orderId);

    List<Order> selectOrderJoinConfig(Integer userId, Integer orderId);

    Integer saveConfig(Config config);

    Config selectConfig(Integer id);
}

OrderServiceImpl

@Service
public class OrderServiceImpl implements OrderService {

    @Autowired
    private OrderMapper orderMapper;

    @Autowired
    private OrderItemMapper orderItemMapper;

    @Autowired
    private ConfigMapper configMapper;

    @Override
    public Integer saveOrder(Order order) {
        return orderMapper.save(order);
    }

    @Override
    public Integer saveOrderItem(OrderItem orderItem, Integer userId) {
        try (HintManager hintManager = HintManager.getInstance()) {
            hintManager.addDatabaseShardingValue("t_order_item", userId);
            return orderItemMapper.save(orderItem);
        }
    }

    @Override
    public Order selectByUserId(Integer userId) {
        return orderMapper.selectByUserId(userId);
    }

    @Override
    public List<Order> selectOrderJoinOrderItem(Integer userId, Integer orderId) {
        try (HintManager hintManager = HintManager.getInstance()) {
            hintManager.addDatabaseShardingValue("t_order_item", userId);
            return orderMapper.selectOrderJoinOrderItem(userId, orderId);
        }
    }

    @Override
    public List<Order> selectOrderJoinOrderItemNoSharding(Integer userId, Integer orderId) {
        return orderMapper.selectOrderJoinOrderItem(userId, orderId);
    }

    @Override
    public List<Order> selectOrderJoinConfig(Integer userId, Integer orderId) {
        return orderMapper.selectOrderJoinConfig(userId, orderId);
    }

    @Override
    public Integer saveConfig(Config config) {
        return configMapper.save(config);
    }

    @Override
    public Config selectConfig(Integer id) {
        return configMapper.selectById(id);
    }
}

Hint实现类

public class HintSharding implements HintShardingAlgorithm<Integer> {

    /**
     *
     * @author hujy
     * @date 2019-09-22 12:23
     * @param availableTargetNames 分片表名的集合
     * @param hintShardingValue 分片键集合
     * @return java.util.Collection<java.lang.String>
     */    
    @Override
    public Collection<String> doSharding(Collection<String> availableTargetNames, HintShardingValue<Integer> hintShardingValue) {
        Collection<String> result = new ArrayList<>();
        for (String each : availableTargetNames) {
            for (Integer value : hintShardingValue.getValues()) {
                if (each.endsWith(String.valueOf(value % 2))) {
                    System.out.println("*********************");
                    result.add(each);
                }
            }
        }
        return result;
    }
}

测试类

@RunWith(SpringRunner.class)
@SpringBootTest
public class ShardingJdbcDemoApplicationTests {

    @Autowired
    private OrderService orderService;

    /**
     * 路由保存
     *
     * @param
     * @return void
     * @author hujy
     * @date 2019-09-20 11:36
     */
    @Test
    public void testSaveOrder() {
        for (int i = 0; i < 10; i++) {
            Integer orderId = 1000 + i;
            Integer userId = 10 + i;

            Order o = new Order();
            o.setOrderId(orderId);
            o.setUserId(userId);
            o.setConfigId(i);
            o.setRemark("save.order");
            orderService.saveOrder(o);

            OrderItem oi = new OrderItem();
            oi.setOrderId(orderId);
            oi.setRemark("save.orderItem");
            orderService.saveOrderItem(oi, userId);
        }
    }

    /**
     * 根据分片键查询
     *
     * @param
     * @return void
     * @author hujy
     * @date 2019-09-20 11:26
     */
    @Test
    public void testSelectByUserId() {
        Integer userId = 12;
        Order o1 = orderService.selectByUserId(userId);
        System.out.println(o1);

        userId = 16;
        Order o2 = orderService.selectByUserId(userId);
        System.out.println(o2);

    }

    /**
     * 与分片子表关联
     *
     * @param
     * @return void
     * @author hujy
     * @date 2019-09-20 11:24
     */
    @Test
    public void testSelectOrderJoinOrderItem() {
        // 指定了子表分片规则
        List<Order> o1 = orderService.selectOrderJoinOrderItem(12, 1002);
        System.out.println(o1);
        // 未指定子表分片规则：导致子表的全路由
        List<Order> o2 = orderService.selectOrderJoinOrderItemNoSharding(12, 1002);
        System.out.println(o2);
    }

    /**
     * 与广播表关联
     *
     * @param
     * @return void
     * @author hujy
     * @date 2019-09-20 11:24
     */
    @Test
    public void testSelectOrderJoinConfig() {
        List<Order> o1 = orderService.selectOrderJoinConfig(12, 1002);
        System.out.println(o1);
        List<Order> o2 = orderService.selectOrderJoinConfig(17, 1007);
        System.out.println(o2);
    }

    /**
     * 广播表保存
     * 对所有数据源进行广播
     *
     * @param
     * @return void
     * @author hujy
     * @date 2019-09-20 11:23
     */
    @Test
    public void testSaveConfig() {
        for (int i = 0; i < 10; i++) {
            Config config = new Config();
            config.setId(i);
            config.setRemark("config " + i);
            orderService.saveConfig(config);
        }
    }

    /**
     * 广播表查询
     * 随机选择数据源
     *
     * @param
     * @return void
     * @author hujy
     * @date 2019-09-20 11:23
     */
    @Test
    public void testSelectConfig() {
        Config config1 = orderService.selectConfig(5);
        System.out.println(config1);

        Config config2 = orderService.selectConfig(7);
        System.out.println(config2);
    }

}

总结

1. 分片表的路由查询条件中必须包含分片键，如t_order$表需要通过user_id分库、order_id分表，t_order_item$表需要通过user_id分库（hint方式强制路由）、order_id分表。

2. 在进行分片表的关联查询时，指定主表的路由键的同时，还要指定子表的路由键，如t_order$关联t_order_item$时，即使查询条件中包含user_id与order_id，可以保证t_order$的正确路由，但由于t_order_item$无法通过user_id进行直接分库，所以仍然需要通过hint方式保证它的强制路由。

3. 一些类似于t_config的配置表如果需要统一的增删改，可以将其配置成广播表，使结果以广播的形式保存（不涉及增删改可以不配置，不影响查询）。查询时会随机选一个数据源进行查询。

完整代码：https://github.com/hjy0319/sharding-jdbc-demo