1. What is ShardingSphere
Positioned as a lightweight Java framework, it provides additional services in Java's JDBC layer. It uses the client to directly connect to the database and provides services in the form of jar packages without additional deployment and dependencies. It can be understood as an enhanced version of the JDBC driver and is fully compatible with JDBC and various ORM frameworks. It consists of three independent products: Sharding-JDBC, Sharding-Proxy and Sharding-Sidecar (planned). They all provide standardized data sharding, distributed transactions, and database governance functions.
2. Application scenarios
With the growth of business, a single database system often cannot meet the application scenarios of massive Internet data, and encounters bottlenecks in terms of performance, availability, and operation and maintenance. In order to solve this problem, the middleware for horizontal and vertical expansion of the database (sub-database and sub-table) is also produced to meet similar needs. Common middleware for sub-database and sub-table include MyCat, ShardingSphere and Alibaba distributed database service DRDS. ShardingSphere is a set of open source distributed database middleware. At present, a detailed introduction to the configuration operation of ShardingSphere's sub-database and sub-table is based on an example based on springBoot+mybatis+mysql.
3. Implementation process
1. Prepare the database environment
1. First install two MySql databases
(their ports are 3306 and 3307 respectively), please refer to here for the creation method .
2. Create databases in the two databases separately
CREATE DATABASE shardingsphere;
3. Create two tables in two databases
DROP TABLE IF EXISTS `t_user_0`;
CREATE TABLE `t_user_0` (
`user_id` bigint(32) NOT NULL COMMENT '主键',
`id_number` varchar(18) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '身份证号码',
`name` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '姓名',
`age` int(4) NULL DEFAULT NULL COMMENT '年龄',
`gender` int(2) NULL DEFAULT 1 COMMENT '性别：1-男；2-女',
`birth_date` date NULL DEFAULT NULL COMMENT '出生日期',
PRIMARY KEY (`user_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci COMMENT = '用户表' ROW_FORMAT = Compact;
DROP TABLE IF EXISTS `t_user_1`;
CREATE TABLE `t_user_1` (
`user_id` bigint(32) NOT NULL COMMENT '主键',
`id_number` varchar(18) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '身份证号码',
`name` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL COMMENT '姓名',
`age` int(4) NULL DEFAULT NULL COMMENT '年龄',
`gender` int(2) NULL DEFAULT 1 COMMENT '性别：1-男；2-女',
`birth_date` date NULL DEFAULT NULL COMMENT '出生日期',
PRIMARY KEY (`user_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci COMMENT = '用户表' ROW_FORMAT = Compact;
2. Code implementation
1. Create a springBoot system
(The IDE tool used in this article is IntelliJ IDEA 2019.2 x64, please search and create it on the Internet for the specific process), and the system name is shardingsphere. First write the pom.xml file of maven. The version used by shardingSphere here is 3.1.0.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.6.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.teamo</groupId>
<artifactId>shardingsphere</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>shardingsphere</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>io.shardingsphere</groupId>
<artifactId>sharding-jdbc-core</artifactId>
<version>3.1.0</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.60</version>
</dependency>
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>4.5.13</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
<version>5.1.32</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid</artifactId>
<version>1.0.9</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>tk.mybatis</groupId>
<artifactId>mapper-spring-boot-starter</artifactId>
<version>2.0.2</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
2. Write the configuration file application.yml file
Here, two data sources ds0 and ds1 are mainly configured, which respectively correspond to the two databases created at the beginning.
server:
port: 8090
spring:
datasource:
type-aliases-package: com.teamo.shardingsphere.mapper
mapper-locations: classpath:/mapper/*.xml
ds0:
url: jdbc:mysql://localhost:3306/shardingsphere?useUnicode=true&characterEncoding=utf-8&useSSL=true
username: root
password:
driver-class-name: com.mysql.jdbc.Driver
initialSize: 2 #初始化大小
maxWait: 6000 #获取连接时最大等待时间，单位毫秒。
min-idle: 5 # 数据库连接池的最小维持连接数
maxActive: 20 # 最大的连接数
initial-size: 5 # 初始化提供的连接数
max-wait-millis: 200 # 等待连接获取的最大超时时间
type: com.alibaba.druid.pool.DruidDataSource
ds1:
url: jdbc:mysql://localhost:3307/shardingsphere?useUnicode=true&characterEncoding=utf-8&useSSL=true
username: root
password:
driver-class-name: com.mysql.jdbc.Driver
initialSize: 2 #初始化大小
maxWait: 6000 #获取连接时最大等待时间，单位毫秒。
min-idle: 5 # 数据库连接池的最小维持连接数
maxActive: 20 # 最大的连接数
initial-size: 5 # 初始化提供的连接数
max-wait-millis: 200 # 等待连接获取的最大超时时间
type: com.alibaba.druid.pool.DruidDataSource
mybatis:
configuration:
map-underscore-to-camel-case: true
type-aliases-package: com.teamo.shardingsphere.model
mapper-locations: classpath:mapper/*.xml
3. Create the data source configuration class DataSourceConfig.java of shardingSphere
Special attention should be paid to the method userRuleConfig, which is to set the sub-database and sub-table rules for the t_user table.
3.1. The t_user table creates four tables in total, which are ds0_t_user0, ds0_t_user1, ds1_t_user0, and ds1_t_user1.
tableRuleConfig.setActualDataNodes(“ds0.t_user_0,ds0.t_user_1,ds1.t_user_0,ds1.t_user_1”);
3.2. The t_user table uses the primary key user_id as the sub-database key and sub-table key
//set t_user sub-database rules
tableRuleConfig.setDatabaseShardingStrategyConfig(
new StandardShardingStrategyConfiguration("user_id", new DatabaseShardingAlgorithm()));
//Set t_user table partition rules
tableRuleConfig.setTableShardingStrategyConfig(
new StandardShardingStrategyConfiguration("user_id", new IdCommonShardingAlgorithm()));
where the class DatabaseShardingAlgorithm is Library class, class IdCommonShardingAlgorithm is divided into Table classes, they all implement the PreciseShardingAlgorithm interface, and implement the corresponding sub-database and sub-table logic in the interface method. For details, please refer to the specific content of these two classes.
package com.teamo.shardingsphere.config;
import com.alibaba.druid.filter.Filter;
import com.alibaba.druid.filter.stat.StatFilter;
import com.alibaba.druid.pool.DruidDataSource;
import com.alibaba.druid.support.http.StatViewServlet;
import com.teamo.shardingsphere.dbstrategy.DatabaseShardingAlgorithm;
import com.teamo.shardingsphere.dbstrategy.GenderTableShardingAlgorithm;
import com.teamo.shardingsphere.dbstrategy.IdCommonShardingAlgorithm;
import io.shardingsphere.api.config.rule.ShardingRuleConfiguration;
import io.shardingsphere.api.config.rule.TableRuleConfiguration;
import io.shardingsphere.api.config.strategy.InlineShardingStrategyConfiguration;
import io.shardingsphere.api.config.strategy.StandardShardingStrategyConfiguration;
import io.shardingsphere.shardingjdbc.api.ShardingDataSourceFactory;
import lombok.Data;
import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.SqlSessionFactoryBean;
import org.mybatis.spring.SqlSessionTemplate;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import javax.sql.DataSource;
import java.sql.SQLException;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.util.concurrent.ConcurrentHashMap;
/***
* @author teamo
* @date 2022-03-18
*/
@Configuration
@Data
public class DataSourceConfig {
@Value("${spring.datasource.type-aliases-package}")
private String typeAliasesPackage;
@Value("${spring.datasource.mapper-locations}")
private String mapperLocation;
@Value("${spring.datasource.ds0.url}")
private String url0;
@Value("${spring.datasource.ds0.username}")
private String userName0;
@Value("${spring.datasource.ds0.password}")
private String password0;
@Value("${spring.datasource.ds0.driver-class-name}")
private String driverClassName0;
@Value("${spring.datasource.ds0.type}")
private String dsType0;
@Value("${spring.datasource.ds1.url}")
private String url1;
@Value("${spring.datasource.ds1.username}")
private String userName1;
@Value("${spring.datasource.ds1.password}")
private String password1;
@Value("${spring.datasource.ds1.driver-class-name}")
private String driverClassName1;
@Value("${spring.datasource.ds1.type}")
private String dsType1;
/**
* 获取数据源Map
*/
private Map<String, DataSource> getDatasourceMap() {
// 真实数据源map
Map<String, DataSource> dataSourceMap = new HashMap<String, DataSource>(2);
// 配置第一个数据源
DruidDataSource dataSource0 = new DruidDataSource();
dataSource0.setDriverClassName(driverClassName0);
dataSource0.setUrl(url0);
dataSource0.setUsername(userName0);
dataSource0.setPassword(password0);
dataSource0.setMinIdle(5);
dataSource0.setMaxActive(20);
dataSource0.setInitialSize(5);
dataSource0.setMaxWait(6000);
dataSourceMap.put("ds0", dataSource0);
// 配置第二个数据源
DruidDataSource dataSource1 = new DruidDataSource();
dataSource1.setDriverClassName(driverClassName1);
dataSource1.setUrl(url1);
dataSource1.setUsername(userName1);
dataSource1.setPassword(password1);
dataSource1.setMinIdle(5);
dataSource1.setMaxActive(20);
dataSource1.setInitialSize(5);
dataSource1.setMaxWait(6000);
dataSourceMap.put("ds1", dataSource1);
return dataSourceMap;
}
@Bean
public Filter statFilter() {
StatFilter filter = new StatFilter();
filter.setSlowSqlMillis(5000);
filter.setLogSlowSql(true);
filter.setMergeSql(true);
return filter;
}
@Bean
public ServletRegistrationBean statViewServlet() {
//创建servlet注册实体
ServletRegistrationBean servletRegistrationBean = new ServletRegistrationBean(new StatViewServlet(), "/druid/*");
//设置ip白名单
servletRegistrationBean.addInitParameter("allow", "127.0.0.1");
//设置控制台管理用户
servletRegistrationBean.addInitParameter("loginUsername", "admin");
servletRegistrationBean.addInitParameter("loginPassword", "123456");
//是否可以重置数据
servletRegistrationBean.addInitParameter("resetEnable", "false");
return servletRegistrationBean;
}
/****
* sharding-jdbc-core（3.1.0）
* @return
* @throws SQLException
*/
@Bean("dataSource")
@Primary
public DataSource dataSource() throws SQLException {
// 配置真实数据源
Map<String, DataSource> dataSourceMap = getDatasourceMap();
// 配置分片规则
ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
// 设置全局默认库
//fshardingRuleConfig.setDefaultDataSourceName("ds0");
shardingRuleConfig.getTableRuleConfigs().add(userRuleConfig());
//设置默认数据源分片策略
shardingRuleConfig.setDefaultDatabaseShardingStrategyConfig(new StandardShardingStrategyConfiguration("user_id", new IdCommonShardingAlgorithm()));
Properties p = new Properties();
p.setProperty("sql.show",Boolean.TRUE.toString());
// 获取数据源对象
DataSource dataSource = ShardingDataSourceFactory.createDataSource(dataSourceMap, shardingRuleConfig, new ConcurrentHashMap(), p);
return dataSource;
}
/**
* 需要手动配置事务管理器
* @param dataSource
* @return
*/
@Bean
public DataSourceTransactionManager transactitonManager(@Qualifier("dataSource") DataSource dataSource){
return new DataSourceTransactionManager(dataSource);
}
@Bean("sqlSessionFactory")
@Primary
public SqlSessionFactory sqlSessionFactory(@Qualifier("dataSource") DataSource dataSource) throws Exception {
SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
bean.setDataSource(dataSource);
bean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath:mapper/*.xml"));
return bean.getObject();
}
@Bean("sqlSessionTemplate")
@Primary
public SqlSessionTemplate sqlSessionTemplate(@Qualifier("sqlSessionFactory") SqlSessionFactory sqlSessionFactory) throws Exception {
return new SqlSessionTemplate(sqlSessionFactory);
}
private TableRuleConfiguration userRuleConfig() {
TableRuleConfiguration tableRuleConfig = new TableRuleConfiguration();
tableRuleConfig.setLogicTable("t_user");
tableRuleConfig.setActualDataNodes("ds0.t_user_0,ds0.t_user_1,ds1.t_user_0,ds1.t_user_1");
tableRuleConfig.setKeyGeneratorColumnName("user_id");
tableRuleConfig.setDatabaseShardingStrategyConfig(
new StandardShardingStrategyConfiguration("user_id", new DatabaseShardingAlgorithm()));
tableRuleConfig.setTableShardingStrategyConfig(
new StandardShardingStrategyConfiguration("user_id", new IdCommonShardingAlgorithm()));
return tableRuleConfig;
}
}
4. Table t_user sub-database rule class DatabaseShardingAlgorithm
The logic of sub-database is the value of the primary key user_id and the value of the remaining 2 to determine the selection of the database.
For example, if a piece of data is currently inserted and the value of the primary key user_id is 3, then (3%2==1) the database selects db1
package com.teamo.shardingsphere.dbstrategy;
import io.shardingsphere.api.algorithm.sharding.PreciseShardingValue;
import io.shardingsphere.api.algorithm.sharding.standard.PreciseShardingAlgorithm;
import java.util.Collection;
/**
* 数据源分片策略
* @author teamo
* @date 2022/3/18
*/
public class DatabaseShardingAlgorithm implements PreciseShardingAlgorithm<Long> {
@Override
public String doSharding(Collection<String> availableTargetNames,
PreciseShardingValue<Long> shardingValue) {
long res = 0 ;
for (String each : availableTargetNames) {
res = shardingValue.getValue() ;
res = res % 2;
if (each.endsWith(res + "")) {
return each;
}
}
throw new UnsupportedOperationException();
}
}
5. Table t_user sub-table rule IdCommonShardingAlgorithm
The table division logic is to take the value of the primary key user_id of the t_user table, the length of user_id is len, take the sum between 0 and len-1 digits, and use the value obtained by the sum value %2 to determine which table to choose. Example 1
: Currently inserting a piece of data, the value of user_id is 11, then the length of user_id is 2, only one digit between 0-1 is 1, then 1%2=1, so the selected library is t_user1 Example 2: Currently inserting a
piece Data, the value of user_id is 131, then the length of user_id is 3, the number between 0-2 is 1 and 3, then (1+3)%2=0, so the selected library is t_user0
package com.teamo.shardingsphere.dbstrategy;
import io.shardingsphere.api.algorithm.sharding.PreciseShardingValue;
import io.shardingsphere.api.algorithm.sharding.standard.PreciseShardingAlgorithm;
import java.util.Collection;
public class IdCommonShardingAlgorithm implements PreciseShardingAlgorithm<Long> {
@Override
public String doSharding(Collection<String> availableTargetNames, PreciseShardingValue<Long> shardingValue) {
String target = availableTargetNames.stream().findFirst().get();
for (String tableName : availableTargetNames) {
if (tableName.endsWith(idToTableSuffix(shardingValue.getValue()))) {
target = tableName;
}
}
return target;
}
private String idToTableSuffix(Long id) {
String idStr = String.valueOf(id);
int total = 0;
for(int i=0;i <idStr.length()-1; i++){
Integer num = Integer.parseInt(idStr.substring(i,i+1));
total+=num;
}
return String.valueOf(total % 2);
}
}
6. Model class
package com.teamo.shardingsphere.model;
import java.io.Serializable;
import java.util.Date;
public class User implements Serializable {
private Long userId;
private String idNumber;
private String name;
private Integer age;
private Integer gender;
private Date birthDate;
private static final long serialVersionUID = 1L;
public Long getUserId() {
return userId;
}
public void setUserId(Long userId) {
this.userId = userId;
}
public String getIdNumber() {
return idNumber;
}
public void setIdNumber(String idNumber) {
this.idNumber = idNumber == null ? null : idNumber.trim();
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name == null ? null : name.trim();
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
public Integer getGender() {
return gender;
}
public void setGender(Integer gender) {
this.gender = gender;
}
public Date getBirthDate() {
return birthDate;
}
public void setBirthDate(Date birthDate) {
this.birthDate = birthDate;
}
}
7. Dao class
package com.teamo.shardingsphere.mapper;
import com.teamo.shardingsphere.model.User;
import org.apache.ibatis.annotations.Mapper;
import org.springframework.stereotype.Component;
import java.util.List;
@Mapper
@Component
public interface UserMapper {
int deleteByPrimaryKey(Long userId);
int insert(User record);
int insertSelective(User record);
User selectByPrimaryKey(Long userId);
int updateByPrimaryKeySelective(User record);
int updateByPrimaryKey(User record);
List<User> selectAll(int start, int limit);
}
8. mapper configuration file (.xml file)
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="com.teamo.shardingsphere.mapper.UserMapper">
<resultMap id="BaseResultMap" type="com.teamo.shardingsphere.model.User">
<id column="user_id" jdbcType="BIGINT" property="userId" />
<result column="id_number" jdbcType="VARCHAR" property="idNumber" />
<result column="name" jdbcType="VARCHAR" property="name" />
<result column="age" jdbcType="INTEGER" property="age" />
<result column="gender" jdbcType="INTEGER" property="gender" />
<result column="birth_date" jdbcType="DATE" property="birthDate" />
</resultMap>
<sql id="Base_Column_List">
user_id, id_number, name, age, gender, birth_date
</sql>
<select id="selectByPrimaryKey" parameterType="java.lang.Long" resultMap="BaseResultMap">
select
<include refid="Base_Column_List" />
from t_user
where user_id = #{userId,jdbcType=BIGINT}
</select>
<delete id="deleteByPrimaryKey" parameterType="java.lang.Long">
delete from t_user
where user_id = #{userId,jdbcType=BIGINT}
</delete>
<insert id="insert" parameterType="com.teamo.shardingsphere.model.User">
insert into t_user (user_id, id_number, name,
age, gender, birth_date
)
values (#{userId,jdbcType=BIGINT}, #{idNumber,jdbcType=VARCHAR}, #{name,jdbcType=VARCHAR},
#{age,jdbcType=INTEGER}, #{gender,jdbcType=INTEGER}, #{birthDate,jdbcType=DATE}
)
</insert>
<insert id="insertSelective" parameterType="com.teamo.shardingsphere.model.User" useGeneratedKeys="true" keyProperty="userId">
insert into t_user
<trim prefix="(" suffix=")" suffixOverrides=",">
<if test="userId != null">
user_id,
</if>
<if test="idNumber != null">
id_number,
</if>
<if test="name != null">
name,
</if>
<if test="age != null">
age,
</if>
<if test="gender != null">
gender,
</if>
<if test="birthDate != null">
birth_date,
</if>
</trim>
<trim prefix="values (" suffix=")" suffixOverrides=",">
<if test="userId != null">
#{userId,jdbcType=BIGINT},
</if>
<if test="idNumber != null">
#{idNumber,jdbcType=VARCHAR},
</if>
<if test="name != null">
#{name,jdbcType=VARCHAR},
</if>
<if test="age != null">
#{age,jdbcType=INTEGER},
</if>
<if test="gender != null">
#{gender,jdbcType=INTEGER},
</if>
<if test="birthDate != null">
#{birthDate,jdbcType=DATE},
</if>
</trim>
</insert>
<update id="updateByPrimaryKeySelective" parameterType="com.teamo.shardingsphere.model.User">
update t_user
<set>
<if test="idNumber != null">
id_number = #{idNumber,jdbcType=VARCHAR},
</if>
<if test="name != null">
name = #{name,jdbcType=VARCHAR},
</if>
<if test="age != null">
age = #{age,jdbcType=INTEGER},
</if>
<if test="gender != null">
gender = #{gender,jdbcType=INTEGER},
</if>
<if test="birthDate != null">
birth_date = #{birthDate,jdbcType=DATE},
</if>
</set>
where user_id = #{userId,jdbcType=BIGINT}
</update>
<update id="updateByPrimaryKey" parameterType="com.teamo.shardingsphere.model.User">
update t_user
set id_number = #{idNumber,jdbcType=VARCHAR},
name = #{name,jdbcType=VARCHAR},
age = #{age,jdbcType=INTEGER},
gender = #{gender,jdbcType=INTEGER},
birth_date = #{birthDate,jdbcType=DATE}
where user_id = #{userId,jdbcType=BIGINT}
</update>
<select id="selectAll" resultMap="BaseResultMap">
select
<include refid="Base_Column_List" />
from t_user
order by user_id
limit #{start}, #{limit}
</select>
</mapper>
9. Service class
Here is a demo example, there is no way to implement the class strictly according to the interface
package com.teamo.shardingsphere.Service;
import com.teamo.shardingsphere.mapper.*;
import com.teamo.shardingsphere.model.*;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@Service
public class BussinessService {
@Autowired
private UserMapper userMapper;
@Transactional
public void saveUser(User user) {
userMapper.insertSelective(user);
}
public List<User> selectAllUser(int start, int limit ){
return userMapper.selectAll(start, limit);
}
}
10. Controller class
The Controller class has written two methods, one is the method of writing t_user table data (using for loop to write 40 records and records), and the other is the method of querying t_user table and paging, the purpose is to detect the data written in t_user table and Whether to operate according to the sub-database and sub-table rules we set when reading data.
package com.teamo.shardingsphere.controller;
import cn.hutool.core.date.DateUtil;
import com.teamo.shardingsphere.Service.BussinessService;
import com.teamo.shardingsphere.model.*;
import com.teamo.shardingsphere.util.SnowflakeKeyGenerator;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.propertyeditors.CustomDateEditor;
import org.springframework.web.bind.WebDataBinder;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@RestController
public class BussinessController {
@Autowired
private BussinessService bussinessService;
@GetMapping("/buss/create")
public String create() {
for (int i = 1; i <= 40; i++) {
User user = new User();
//user.setUserId(snowflakeKeyGenerator.nextId());
user.setUserId(Long.parseLong((i)+""));
user.setName("王小晖" + i);
user.setGender(GenderEnum.MALE.getCode());
user.setAge(20 + i);
user.setBirthDate(DateUtil.parseDate("1989-08-16"));
user.setIdNumber("4101231989691" + i);
bussinessService.saveUser(user);
}
return "成功";
}
@GetMapping("/buss/allUser")
public List<User> findAllUser(int start, int limit){
return bussinessService.selectAllUser(start, limit);
}
}
11. SpringBoot startup class
package com.teamo.shardingsphere;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
@SpringBootApplication(exclude={
DataSourceAutoConfiguration.class})
public class ShardingsphereApplication {
public static void main(String[] args) {
SpringApplication.run(ShardingsphereApplication.class, args);
}
}
At this point, the entire demo system is written
3. Run the test
1. Start the system
Right-click the startup class to start the demo system
2. Data insertion test
After the system starts successfully, open the browser, and enter http://localhost:9090/buss/create in the address bar. When the page is displayed successfully, it indicates that the method of inserting the data in the t_user table is executed successfully.
Then log in to the database and check the input insertion into the database table
ds0 library: Extract the third record of the t_user0 and t_user1 tables of the ds0 library to perform logical verification of the
third record of t_user0 user_id=6
The value of the user_id of the sub-database rule %2==0, so the length of the user_id value of the sub-database to ds0
sub-table rule is 1, and there is no number between 0 and (1-1), which is 0, so the sub-table is divided into t_user0, so the record with
user_id=6 will be inserted In the ds0 library, in the t_user0 table, the verification is correct
The user_id=14 of the third record of t_user1
is the value of user_id in the sub-database rule %2==0, so
the length of the user_id value of the sub-database to ds0 sub-table rule is 2, and the value between [0 and (2-1)) is 1, press The sum of the lengths is 1, so the table is divided into t_user1,
so the record with user_id=6 will be inserted into the ds0 library, t_user1 table, and the verification is correct
ds1 library
ds1 library: Extract the 5th record of the t_user0 and t_user1 tables of the ds0 library for sub-database sub-table logic verification. User_id=9 of the
fifth record of t_user0
The value of user_id in the sub-library rule %2==1, so the sub-library The length of the user_id value of the ds1
sub-table rule is 1, and there is no number between 0 and (1-1), which is 0, so the sub-table is assigned to t_user0, so the record with
user_id=6 will be inserted into the ds1 library, t_user0 table, and the calibration Verify correct
The user_id=19 of the 5th record of t_user1
is the value of user_id of the sub-database rule %2==1, so the
length of the user_id value of the sub-database to ds1 sub-table rule is 2, and the value between [0 and (2-1)) is 1, press The sum of the lengths is 1, so the table is divided into t_user1,
so the record with user_id=6 will be inserted into the ds1 library, t_user1 table, and the verification is correct
3. Data query test
Enter http://localhost:9090/buss/allUser?start=0&limit=5 in the browser
to view the sql log printed in the background. After the system parses the rules of ShardingSphere, the sql actually sent to the database is
It can be seen that after ShardingSphere performs sub-database and sub-table configuration operations, the data source of the system's single-table query will be obtained from all configured database tables (in this case, two databases and four tables).
Four. Summary
When we plan the system at the beginning, if we foresee that the system may have performance risks caused by large data storage, or during the operation and maintenance process, the system finds that the data volume of a certain table in a certain business is too large to cause performance problems. Consider adopting ShardingSphere. The key to efficient use of ShardingSphere is the determination and setting of sub-database and sub-table rules. The determination of these rules needs to be determined according to the logic of the query business of the table. Only a good rule setting can make it show its advantages. The optimal rule for sub-database sub-table key is to split the corresponding data into different tables according to the business, such as splitting by unit (the premise is that the business is carried out in the same unit, and there are fewer businesses crossing units), This will have the desired effect.