SpringBoot+Sharding-JDBC operation sub-database and table

What is Sharding-JDBC? What is sub-database sub-table? Why sub-database and sub-table?

You can view this blog:

Apache-ShardingSphere (distributed database middleware, operation tool for sub-database and table)

Sharding-JDBC operation level sub-table

1. Build the environment

Basic environment: SpringBoot2.2.1 + MybatisPlus + Sharding-JDBC + Druid connection pool
Create SpringBoot project
Modify the SpringBoot project version to 2.2.1

Introduce related dependencies

<dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <!-- Druid连接池 -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid</artifactId>
            <version>1.1.20</version>
        </dependency>
        <!-- Mysql驱动依赖 -->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
        </dependency>
        <!-- MybatisPlus -->
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
            <version>3.0.5</version>
        </dependency>
        <!-- Sharding-JDBC -->
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
            <version>4.0.0-RC1</version>
        </dependency>

        <!-- lombok -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>

    </dependencies>

Second, create databases and database tables in the way of horizontal sub-tables

Create a database goods_db;
Create tables goods_1, goods_2 in goods_db;
The agreed rule is to add the data to goods_1 if the added product id is an even number, and add the data to goods_2 if it is an even number;
Structure display

Three, write code

Create Goods entity class

@Data
public class Goods {
    private Long gid;
    private String gname;
    private Long userId;
    private String gstatus;
}

The content generated by the @Data annotation is as follows:

Create GoodsMapper

@Repository
public interface GoodsMapper extends BaseMapper<Goods> {
}

Start class configuration scan Mapper package

@SpringBootApplication
@MapperScan("com.xxx.mapper")
public class ShardingJdbcApplication {

    public static void main(String[] args) {
        SpringApplication.run(ShardingJdbcApplication.class, args);
    }

}

Four, application.properties configuration Sharding-JDBC

# 配置Sharding-JDBC的分片策略
# 配置数据源，给数据源起名g1,g2...此处可配置多数据源
spring.shardingsphere.datasource.names=g1
# 配置数据源具体内容————————包含连接池，驱动，地址，用户名，密码
# 由于上面配置数据源只有g1因此下面只配置g1.type,g1.driver-class-name,g1.url,g1.username,g1.password
spring.shardingsphere.datasource.g1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g1.url=jdbc:mysql://localhost:3306/goods_db?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g1.username=root
spring.shardingsphere.datasource.g1.password=123456
# 配置表的分布，表的策略
spring.shardingsphere.sharding.tables.goods.actual-data-nodes=g1.goods_$->{1..2}
# 指定goods表主键gid 生成策略为 SNOWFLAKE
spring.shardingsphere.sharding.tables.goods.key-generator.column=gid
spring.shardingsphere.sharding.tables.goods.key-generator.type=SNOWFLAKE
# 指定分片策略约定gid值是偶数添加到goods_1表，如果gid是奇数添加到goods_2表
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.sharding-column=gid
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.algorithm-expression=goods_$->{gid % 2 + 1}
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true

PS: Such configuration files can be found on the ShardingShpere official website, so don't worry about not remembering them.

Five, write test code

ShardingjdbcdemoApplicationTests.Java code:

package com.ws.shardingjdbcdemo;
import com.ws.shardingjdbcdemo.mapper.GoodsMapper;
import com.ws.shardingjdbcdemo.pojo.Goods;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
@SpringBootTest
class ShardingjdbcdemoApplicationTests {
@Autowired
GoodsMapper goodsMapper;
@Test
void addGoods() {
Goods good = new Goods();
good.setGname("小米手机");
good.setUserId(100L);
good.setGstatus("已发布");
goodsMapper.insert(good);
}
}

I started the test method, and I got an error:

It probably means that two beans with the same name are not allowed.

solution:

spring.main.allow-bean-definition-overriding=true

Configure this in application.properties

# 配置Sharding-JDBC的分片策略
# 配置数据源，给数据源起名g1,g2...此处可配置多数据源
spring.shardingsphere.datasource.names=g1
# 配置允许后面的Bean覆盖前面名称重复的Bean
spring.main.allow-bean-definition-overriding=true
# 配置数据源具体内容————————包含连接池，驱动，地址，用户名，密码
# 由于上面配置数据源只有g1因此下面只配置g1.type,g1.driver-class-name,g1.url,g1.username,g1.password
spring.shardingsphere.datasource.g1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g1.url=jdbc:mysql://localhost:3306/goods_db?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g1.username=root
spring.shardingsphere.datasource.g1.password=123456
# 配置表的分布，表的策略
spring.shardingsphere.sharding.tables.goods.actual-data-nodes=g1.goods_$->{1..2}
# 指定goods表主键gid 生成策略为 SNOWFLAKE
spring.shardingsphere.sharding.tables.goods.key-generator.column=gid
spring.shardingsphere.sharding.tables.goods.key-generator.type=SNOWFLAKE
# 指定分片策略约定gid值是偶数添加到goods_1表，如果gid是奇数添加到goods_2表
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.sharding-column=gid
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.algorithm-expression=goods_$->{gid % 2 + 1}
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true

Start the test method again to successfully insert data and print out the SQL statement on the console:

goods_1 table:

goods_2 table:

PS: You can see that the gid is odd, data is inserted in goods_2, and there is nothing in goods_1.

Console:

Next, test whether the table is inserted according to odd and even numbers.

Modify the code of the addGoods() method:

@Test
void addGoods() {
for (int i = 0; i < 10; i++){
Goods good = new Goods();
good.setGname("小米手机" + i);
good.setUserId(100L);
good.setGstatus("已发布");
goodsMapper.insert(good);
}
}

Results of the:

goods_1 table:

goods_2 table:

It can be seen that the rules of inserting 2 tables for odd numbers and 1 table for even numbers are completely followed.

Test query:

1. Write the getGood() method

@Test
void getGood(){
QueryWrapper<Goods> queryWrapper = new QueryWrapper<>();
queryWrapper.eq("gid",479688915029065729L);
Goods good = goodsMapper.selectOne(queryWrapper);
System.out.println(good.toString());
}

2. Execute getGood(), and successfully print out the data information of gid = 479688915029065729

The above is the operation of Sharding-JDBC on the horizontal sub-table. If you have any questions, you can leave a comment, aiming to learn together and make progress together!

The above describes how to use Sharding-JDBC to operate the horizontal sub-table, and then play with the horizontal sub-database sub-table operation.

Sharding-JDBC operation level sub-database + level sub-table

1. Create two databases goods_db_1 and goods_db_2, each database contains two tables goods_1 and goods_2, goods_1 and goods_2 have the same structure as the above sub-table.

2. Configure application.properties

# 配置Sharding-JDBC的分片策略
# 配置数据源，给数据源起名g1,g2...此处可配置多数据源
spring.shardingsphere.datasource.names=g1,g2
# 配置允许一个实体类映射多张表
spring.main.allow-bean-definition-overriding=true
# 配置数据源具体内容————————包含连接池，驱动，地址，用户名，密码
# g1配置g1.type,g1.driver-class-name,g1.url,g1.username,g1.password
spring.shardingsphere.datasource.g1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g1.url=jdbc:mysql://localhost:3306/goods_db_1?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g1.username=root
spring.shardingsphere.datasource.g1.password=123456
# 配置数据源具体内容————————包含连接池，驱动，地址，用户名，密码
# g2配置g2.type,g2.driver-class-name,g2.url,g2.username,g2.password
spring.shardingsphere.datasource.g2.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g2.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g2.url=jdbc:mysql://localhost:3306/goods_db_2?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g2.username=root
spring.shardingsphere.datasource.g2.password=123456
# 配置数据库的分布，表的分布
# m1:goods_1 goods_2; m2:goods_1,goods_2;
spring.shardingsphere.sharding.tables.goods.actual-data-nodes=g$->{1..2}.goods_$->{1..2}
# 指定goods表主键gid 生成策略为 SNOWFLAKE
spring.shardingsphere.sharding.tables.goods.key-generator.column=gid
spring.shardingsphere.sharding.tables.goods.key-generator.type=SNOWFLAKE
# 指定数据库分片策略约定user_id值是偶数添加到goods_db_1中，奇数添加到goods_db_2中
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.sharding-column=user_id
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.algorithm-expression=g$->{user_id % 2 + 1}
# 指定表分片策略约定gid值是偶数添加到goods_1表，如果gid是奇数添加到goods_2表
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.sharding-column=gid
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.algorithm-expression=goods_$->{gid % 2 + 1}
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true

3. Write the test method addGoods02()

@Test
void addGoods02(){
Goods good = new Goods();
good.setGname("华为手机");
good.setUserId(100L);
good.setGstatus("已发布");
goodsMapper.insert(good);
}

4. Execute addGoods02() method

Since our user_id is set to 100L, which is an even number, according to our original intention, the even number is stored in goods_db_1.

5. Implementation results

The console sql statement is consistent with what we expect.

The database results are also consistent with our expectations.

You can try it yourself for the remaining batch insert, query, modify, and delete operations.

Sharding-JDBC operation vertical sub-database + horizontal sub-database + horizontal sub-table

Background description:

In the sub-database sub-table & sub-database sub-table tool article, I learned what is vertical sub-database sub-table, horizontal sub-database sub-table, and why sub-library sub-table; in our actual project, there will be multiple databases after vertical sub-database , Each database has its own tables and public tables (operating public tables will be described later in this section).

In the above chapter, we have used Sharding-JDBC to operate level database and table. On the basis of the above, we also have a database user_db, which contains a t_user table to store user information. There are multiple in one of our projects. Database, each database has different tables.

The database structure and t_user table structure are as follows:

1. Demand analysis

There are so many databases and tables above in our project. When we operate the t_user table, we can quickly and accurately operate to the userdb database> t_user table.

Second, realize the demand

1. Configure the application.properties file

On the basis of the above horizontal sub-database and sub-tables, write the configuration about user_db and t_user tables.

# 配置Sharding-JDBC的分片策略
# 配置数据源，给数据源起名g1,g2...此处可配置多数据源
spring.shardingsphere.datasource.names=g1,g2,u0
# 配置允许一个实体类映射多张表
spring.main.allow-bean-definition-overriding=true
# 配置g1数据源具体内容————————包含连接池，驱动，地址，用户名，密码
# g1配置g1.type,g1.driver-class-name,g1.url,g1.username,g1.password
spring.shardingsphere.datasource.g1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g1.url=jdbc:mysql://localhost:3306/goods_db_1?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g1.username=root
spring.shardingsphere.datasource.g1.password=123456
# 配置g2数据源具体内容————————包含连接池，驱动，地址，用户名，密码
# g2配置g2.type,g2.driver-class-name,g2.url,g2.username,g2.password
spring.shardingsphere.datasource.g2.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g2.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g2.url=jdbc:mysql://localhost:3306/goods_db_2?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g2.username=root
spring.shardingsphere.datasource.g2.password=123456
# 配置u0数据源具体内容————————包含连接池，驱动，地址，用户名，密码
# u0配置u0.type,u0.driver-class-name,u0.url,u0.username,u0.password
spring.shardingsphere.datasource.u0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.u0.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.u0.url=jdbc:mysql://localhost:3306/user_db?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.u0.username=root
spring.shardingsphere.datasource.u0.password=123456
#配置垂直分库t_user的策略
spring.shardingsphere.sharding.tables.t_user.actual-data-nodes=u0.t_user
spring.shardingsphere.sharding.tables.t_user.key-generator.column=user_id
spring.shardingsphere.sharding.tables.t_user.key-generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.sharding-column=user_id
#由于只有一张表，因此在此直接写表明，不需要像水平分多个表那样写策略
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.algorithm-expression=t_user
# 配置数据库的分布，表的分布
# g1:goods_1 goods_2; g2:goods_1,goods_2;
spring.shardingsphere.sharding.tables.goods.actual-data-nodes=g$->{1..2}.goods_$->{1..2}
# 指定goods表主键gid 生成策略为 SNOWFLAKE
spring.shardingsphere.sharding.tables.goods.key-generator.column=gid
spring.shardingsphere.sharding.tables.goods.key-generator.type=SNOWFLAKE
# 指定数据库分片策略约定user_id值是偶数添加到goods_db_1中，奇数添加到goods_db_2中
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.sharding-column=user_id
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.algorithm-expression=g$->{user_id % 2 + 1}
# 指定表分片策略约定gid值是偶数添加到goods_1表，如果gid是奇数添加到goods_2表
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.sharding-column=gid
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.algorithm-expression=goods_$->{gid % 2 + 1}
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true

2. Write the code

User.java

package com.ws.shardingjdbcdemo.pojo;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
@Data
@TableName("t_user")
public class User {
private Long userId;
private String username;
private String ustatus;
}

UserMapper.java

package com.ws.shardingjdbcdemo.mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.ws.shardingjdbcdemo.pojo.User;
import org.springframework.stereotype.Repository;
@Repository
public interface UserMapper extends BaseMapper<User> {
}

Tests.java

@Test
void addUser(){
User user = new User();
user.setUsername("琳妹妹");
user.setUstatus("0");
userMapper.insert(user);
}
@Test
void getUser(){
QueryWrapper<User> queryWrapper = new QueryWrapper<>();
//此处请填写自己程序生成的ID
queryWrapper.eq("user_id",100L);
User good = userMapper.selectOne(queryWrapper);
System.out.println(good.toString());
}

3. Test and results

addUser() result:

getUser result:

Sharding-JDBC operation common table

Background description:

In the project, there are generally some tables whose contents are fixed, or tables that are rarely modified, but they are often queried in association with these tables, such as some status information. Generally in our project this kind of table will be stored in our various databases, so it is called a public table.

In the above chapter, we used Sharding-JDBC to operate the vertical sub-database + horizontal sub-database + horizontal sub-table, assuming that there is a common table t_dict in each of our databases.

Database & table structure:

demand:

When operating a common table, such as adding and deleting operations, this table in all databases will be modified.

achieve:

1. Based on the previous chapter, write and configure application.properties (configuration common table part)

# 配置Sharding-JDBC的分片策略
# 配置数据源，给数据源起名g1,g2...此处可配置多数据源
spring.shardingsphere.datasource.names=g1,g2,u0
# 配置允许一个实体类映射多张表
spring.main.allow-bean-definition-overriding=true
# 配置g1数据源具体内容————————包含连接池，驱动，地址，用户名，密码
# g1配置g1.type,g1.driver-class-name,g1.url,g1.username,g1.password
spring.shardingsphere.datasource.g1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g1.url=jdbc:mysql://localhost:3306/goods_db_1?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g1.username=root
spring.shardingsphere.datasource.g1.password=123456
# 配置g2数据源具体内容————————包含连接池，驱动，地址，用户名，密码
# g2配置g2.type,g2.driver-class-name,g2.url,g2.username,g2.password
spring.shardingsphere.datasource.g2.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g2.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g2.url=jdbc:mysql://localhost:3306/goods_db_2?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g2.username=root
spring.shardingsphere.datasource.g2.password=123456
# 配置u0数据源具体内容————————包含连接池，驱动，地址，用户名，密码
# u0配置u0.type,u0.driver-class-name,u0.url,u0.username,u0.password
spring.shardingsphere.datasource.u0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.u0.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.u0.url=jdbc:mysql://localhost:3306/user_db?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.u0.username=root
spring.shardingsphere.datasource.u0.password=123456
# 配置公共表
spring.shardingsphere.sharding.broadcast-tables=t_dict
# 配置公共表ID及生成策略
spring.shardingsphere.sharding.tables.t_dict.key-generator.column=dict_id
spring.shardingsphere.sharding.tables.t_dict.key-generator.type=SNOWFLAKE
#配置垂直分库t_user的策略
spring.shardingsphere.sharding.tables.t_user.actual-data-nodes=u0.t_user
spring.shardingsphere.sharding.tables.t_user.key-generator.column=user_id
spring.shardingsphere.sharding.tables.t_user.key-generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.sharding-column=user_id
#由于只有一张表，因此在此直接写表明，不需要像水平分多个表那样写策略
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.algorithm-expression=t_user
# 配置数据库的分布，表的分布
# g1:goods_1 goods_2; g2:goods_1,goods_2;
spring.shardingsphere.sharding.tables.goods.actual-data-nodes=g$->{1..2}.goods_$->{1..2}
# 指定goods表主键gid 生成策略为 SNOWFLAKE
spring.shardingsphere.sharding.tables.goods.key-generator.column=gid
spring.shardingsphere.sharding.tables.goods.key-generator.type=SNOWFLAKE
# 指定数据库分片策略约定user_id值是偶数添加到goods_db_1中，奇数添加到goods_db_2中
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.sharding-column=user_id
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.algorithm-expression=g$->{user_id % 2 + 1}
# 指定表分片策略约定gid值是偶数添加到goods_1表，如果gid是奇数添加到goods_2表
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.sharding-column=gid
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.algorithm-expression=goods_$->{gid % 2 + 1}
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true

2. Write code

MyDict.java

package com.ws.shardingjdbcdemo.pojo;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
@Data
@TableName("t_dict")
public class MyDict {
private Long dictId;
private String dictName;
private String dictCode;
}

MyDictMapper.java

package com.ws.shardingjdbcdemo.mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.ws.shardingjdbcdemo.pojo.MyDict;
import org.springframework.stereotype.Repository;
@Repository
public interface MyDictMapper extends BaseMapper<MyDict> {
}

Tests.java

/**
* 下面是公共表测试方法
*/
@Test
void addDict(){
MyDict myDict = new MyDict();
myDict.setDictName("已启用");
myDict.setDictCode("1");
myDictMapper.insert(myDict);
}
@Test
void deleteDict(){
QueryWrapper<MyDict> wrapper = new QueryWrapper<>();
wrapper.eq("dict_id","");
myDictMapper.delete(wrapper);
}

3. Test

addDict() method:

deleteDict() method:

The above is the operation of Sharding-JDBC of Apache-ShardingSphere on sub-database sub-tables and public tables. If you have any questions, you can leave a comment below, aiming to learn together and make progress together!