Before we talk about an article in a separate read and write Mysql8 the (end of the article there is a link) based on the implementation process for the said sub-library sub-table.
Interpretation of the Concept
Vertical slices
In accordance with the split service is called a vertical slice, also known as longitudinal splits, its core idea is for special purposes. Before the split, a database composed of a plurality of data tables, each table corresponding to different services. After the split, the table is classified according to the service will be distributed to different databases, so that the pressure of the dispersion to a different database. The following figure shows the business needs, the user table and the Order table to a different vertical fragmentation scheme databases.
Vertical slice architecture and design often requires adjustment. Generally, the Internet is too late to respond to rapidly changing business needs; moreover, it also does not really solve the bottleneck of a single point. Vertical split can alleviate the problem and bring the amount of data traffic, but can not cure. After the split vertically if the amount of data in the table exceeds the threshold still can carry a single node, it is necessary to further process the slice level.
Slice level
Also known as horizontal fragmentation lateral resolution. Relative to a vertical slice, it is no longer classified data according to the service logic, but through a field (or a few fields), data according to certain rules or dispersed into a plurality of database tables, each fragment comprising only part of the data. For example: The primary key slice, the primary key into the even-numbered recording libraries 0 (or table), the odd-numbered records into a database primary key (or table), as shown below.
The level of fragmentation theoretically break through the bottleneck of single amounts of data processing, and extended relative freedom, is sub-standard library sub-table solutions.
Development Readiness
Sub-library sub-table common components is shardingsphere, now is the top-level apache project, this time we use springboot2.1.9 + shardingsphere4.0.0-RC2 (both latest version) to complete the operation sub-library sub-table.
Suppose an Orders table, we need to divide it into two banks, each of three tables, determine the location of the final modulo data according to id field, the database environment configured as follows:
- 172.31.0.129
- blog
- t_order_0
- t_order_1
- t_order_2
- blog
- 172.31.0.131
- blog
- t_order_0
- t_order_1
- t_order_2
- blog
Logic table three tables for t_order, you can prepare all the other data table based on construction of the table statement.
DROP TABLE IF EXISTS `t_order_0;
CREATE TABLE `t_order_0` (
`id` bigint(20) NOT NULL,
`name` varchar(255) DEFAULT NULL COMMENT '名称',
`type` varchar(255) DEFAULT NULL COMMENT '类型',
`gmt_create` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;Note, do not rule generated from the primary key is set to grow, according to certain rules need to generate the primary key, used here shardingsphere in SNOWFLAKE commonly known as the snow algorithm to generate the primary key
Code
- Modify pom.xml, components related to the introduction
<properties>
<java.version>1.8</java.version>
<mybatis-plus.version>3.1.1</mybatis-plus.version>
<sharding-sphere.version>4.0.0-RC2</sharding-sphere.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>2.0.1</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.15</version>
</dependency>
<dependency>
<groupId>com.baomidou</groupId>
<artifactId>mybatis-plus-boot-starter</artifactId>
<version>${mybatis-plus.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>${sharding-sphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-namespace</artifactId>
<version>${sharding-sphere.version}</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>- Configure mysql-plus
@Configuration
@MapperScan("com.github.jianzh5.blog.mapper")
public class MybatisPlusConfig {
/**
* 攻击 SQL 阻断解析器
*/
@Bean
public PaginationInterceptor paginationInterceptor(){
PaginationInterceptor paginationInterceptor = new PaginationInterceptor();
List<ISqlParser> sqlParserList = new ArrayList<>();
sqlParserList.add(new BlockAttackSqlParser());
paginationInterceptor.setSqlParserList(sqlParserList);
return new PaginationInterceptor();
}
/**
* SQL执行效率插件
*/
@Bean
// @Profile({"dev","test"})
public PerformanceInterceptor performanceInterceptor() {
return new PerformanceInterceptor();
}
}- Write entity class Order
@Data
@TableName("t_order")
public class Order {
private Long id;
private String name;
private String type;
private Date gmtCreate;
}- Write DAO layer, OrderMapper
/**
* 订单Dao层
*/
public interface OrderMapper extends BaseMapper<Order> {
}- Write interfaces and interface
public interface OrderService extends IService<Order> {
}
/**
* 订单实现层
* @author jianzh5
* @date 2019/10/15 17:05
*/
@Service
public class OrderServiceImpl extends ServiceImpl<OrderMapper, Order> implements OrderService {
}- Profiles (configuration instructions see note)
server.port=8080
# 配置ds0 和ds1两个数据源
spring.shardingsphere.datasource.names = ds0,ds1
#ds0 配置
spring.shardingsphere.datasource.ds0.type = com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds0.driver-class-name = com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.ds0.jdbc-url = jdbc:mysql://192.168.249.129:3306/blog?characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false
spring.shardingsphere.datasource.ds0.username = root
spring.shardingsphere.datasource.ds0.password = 000000
#ds1 配置
spring.shardingsphere.datasource.ds1.type = com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds1.driver-class-name = com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.ds1.jdbc-url = jdbc:mysql://192.168.249.131:3306/blog?characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false
spring.shardingsphere.datasource.ds1.username = root
spring.shardingsphere.datasource.ds1.password = 000000
# 分库策略 根据id取模确定数据进哪个数据库
spring.shardingsphere.sharding.default-database-strategy.inline.sharding-column = id
spring.shardingsphere.sharding.default-database-strategy.inline.algorithm-expression = ds$->{id % 2}
# 具体分表策略
# 节点 ds0.t_order_0,ds0.t_order_1,ds1.t_order_0,ds1.t_order_1
spring.shardingsphere.sharding.tables.t_order.actual-data-nodes = ds$->{0..1}.t_order_$->{0..2}
# 分表字段id
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.sharding-column = id
# 分表策略 根据id取模,确定数据最终落在那个表中
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.algorithm-expression = t_order_$->{id % 3}
# 使用SNOWFLAKE算法生成主键
spring.shardingsphere.sharding.tables.t_order.key-generator.column = id
spring.shardingsphere.sharding.tables.t_order.key-generator.type = SNOWFLAKE
#spring.shardingsphere.sharding.binding-tables=t_order
spring.shardingsphere.props.sql.show = true- Write unit tests to see whether the correct results
public class OrderServiceImplTest extends BlogApplicationTests {
@Autowired
private OrderService orderService;
@Test
public void testSave(){
for (int i = 0 ; i< 100 ; i++){
Order order = new Order();
order.setName("电脑"+i);
order.setType("办公");
orderService.save(order);
}
}
@Test
public void testGetById(){
long id = 1184489163202789377L;
Order order = orderService.getById(id);
System.out.println(order.toString());
}
}See data in the data table, the data confirm that normal insertion
So far developed sub-library sub-table
Past Events
Welcome to my personal public concern number: JAVA Rizhilu