Sharding-JDBC achieve the level of resolution - single reservoir sub-table

Reference: ape world https://mp.weixin.qq.com/s/901rNhc4WhLCQ023zujRVQ Author: Yin Ji Huan

　　When a single sharp rise in the number of tables, more than 1 million or more, this time it should split the table level.

What is the level splitting table?

　　The table is split into N a table, just like a big stone not move, and then cut 10, such a transfer can be moved. The principle is the same. In addition to the number of pressure balancing, but also to spread the pressure read and write requests, of course, this depends on your slice algorithm, the algorithm is able to make reasonably uniform distribution of data and improve performance. Today, we are mainly talking about splitting a table in a single database, that is, regardless of the library, only points table.

　　a user table from the original is split into four, data is evenly distributed in the three tables, i.e. the original user = user0 + user1 + user2 + user3.

　　Technology selection: SpringBoot + Sharding-JDBC + MyBatis

1. Core Jar package

　　With the vertical split

2.  YML file configuration

# Set of data source name, data source configuration corresponding to the following name 
Spring: 
  main: 
    the allow -bean-Definition-overriding: to true 
  shardingsphere: 
    DataSource: 
      names: DB1 
      # primary data source 
      DB1: 
        type: com.alibaba.druid.pool.DruidDataSource 
        Driver - class - name: com.mysql.cj.jdbc.Driver 
        url: jdbc: MySQL: // ? localhost: 3306 / DB_USER characterEncoding = UTF-8 
        username: **** 
        password: **** 
    Sharding: 
      the Tables: 
        User: 
          # partition table configuration 
          Actual -data-nodes.db1: User $ {_0 .. . 3 } 
          # inline expression 
          Table -strategy.inline.sharding- column: ID 
          Table -strategy.inline.algorithm-expression The: User id.longValue {$ _ ()% . 4 } 
    The props: 
      # Open SQL display, default false 
      SQL: 
        Show: to true

• actual-data-nodes configuration points table information, inline expression used here, which translates to db1.user0, db1.user1, db1.user2, db1.user3
• Field inline.sharding-column sub-table, with the side part table id
• inline.algorithm-expression part table row arithmetic expression grammar must meet groovy, the above configuration is to use the modulo fragment id

 

　　If we have more complex needs of fragmentation, fragmentation can customize the algorithm to achieve:

Sharding: 
      Tables: 
        User: 
          # min Field 
          Table -strategy.standard.sharding- column: ID 
          # custom algorithm based part tables 
          Table -strategy.standard.precise-algorithm- class -name: COM * * MyPreciseShardingAlgorithm...

　　Algorithms categories:

public class MyPreciseShardingAlgorithm implements PreciseShardingAlgorithm<Long> {

    @Override
    public String doSharding(Collection<String> availableTargetNames, PreciseShardingValue<Long> shardingValue) {
        for (String tableName : availableTargetNames) {
            if (tableName.endsWith(shardingValue.getValue() % 4 + "")) {
                return tableName;
            }
        }
        throw new IllegalArgumentException();
    }

}

　　In doSharding method you can do some processing based on the parameters shardingValue, eventually returning this data table name can be fragmented.

　　In addition to single field fragmentation, fragmentation also supports multi-column, you can see for yourself what the document operation.

　　Need to be part table configuration, no part table no configuration database operations do not change the code line.

　　If we want a single library based on the sub-table, do separate read and write, the same is very simple, as long as the configuration data from a source can be, configured as follows:

spring.shardingsphere.datasource.names=master,slave

# 主数据源
spring.shardingsphere.datasource.master.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.master.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.master.url=jdbc:mysql://localhost:3306/ds_0?characterEncoding=utf-8
spring.shardingsphere.datasource.master.username=root
spring.shardingsphere.datasource.master.password=123456

# 从数据源
spring.shardingsphere.datasource.slave.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.slave.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.slave.url=jdbc:mysql://localhost:3306/ds_1?characterEncoding=utf-8
spring.shardingsphere.datasource.slave.username=root
spring.shardingsphere.datasource.slave.password=123456

# 分表配置
spring.shardingsphere.sharding.tables.user.actual-data-nodes=ds0.user_${0..3}
spring.shardingsphere.sharding.tables.user.table-strategy.inline.sharding-column=id
spring.shardingsphere.sharding.tables.user.table-strategy.inline.algorithm-expression=user_${id.longValue()%4}

# 读写分离配置
spring.shardingsphere.sharding.master-slave-rules.ds0.master-data-source-name=master
spring.shardingsphere.sharding.master-slave-rules.ds0.slave-data-source-names=slave