[Sharding-JDBC Series 1] Spring Boot integrates Sharding-JDBC to achieve separation of reading and writing

table of Contents

Introduction to Sharding-JDBC

Build mysql master-slave database

Springboot application construction

pom dependency

application.properties configuration

test

Enable log (not required)

---

Introduction to Sharding-JDBC

ShardingSphere-JDBC is positioned as a lightweight distributed database middleware solution that provides additional services in the JDBC layer of Java. It uses the client to directly connect to the database and provides services in the form of jar packages without additional deployment and dependencies. It can be understood as an enhanced version of the JDBC driver, fully compatible with JDBC and various ORM frameworks.

• Applicable to any JDBC-based ORM framework, such as: JPA, Hibernate, Mybatis, Spring JDBC Template or directly using JDBC.
• Support any third-party database connection pool, such as: DBCP, C3P0, BoneCP, Druid, HikariCP, etc.
• It supports any database that implements the JDBC specification, and currently supports MySQL, Oracle, SQLServer, PostgreSQL and any database that follows the SQL92 standard.

Features:

• Sub-library & sub-table
• Read and write separation
• Sharding strategy customization
• Decentralized distributed primary key
• Distributed governance
• Flexible transaction
• Standardized transaction interface

Build mysql master-slave database

You can refer to the previous article, "[Mycat Series One] Detailed Tutorial of Building MySQL Master-Slave Replication Based on Docker" to build a master and two-slave mysql, of course, you can also use multiple linux to build without using docker, the details are as follows:

Database type	database	IP in docker container	Host IP (external IP)
Main: mysql-master	test	172.17.0.2:3306	192.168.239.128:3307
From: mysql-slave1	test	172.17.0.3:3306	192.168.239.128:3308
From: mysql-slave2	test	172.17.0.4:3306	192.168.239.128:3309

After configuring the master and slave, create a new database test in the master library. You can use third-party client tools such as Navicat, or command line scripts. The SQL scripts are as follows:

USE `test`;

DROP TABLE IF EXISTS `t_user`;

CREATE TABLE `t_user` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `name` varchar(100) DEFAULT '' COMMENT '名称',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1307873057269878786 DEFAULT CHARSET=utf8 COMMENT='用户表';

Navicat effects are as follows:

Springboot application construction

This article will integrate SpringBoot + Mybatis-plus + Druid + Sharding-JDBC + MySQL , as follows:

pom dependency

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.38</version>
        </dependency>

        <!-- 数据库连接池 -->
       <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid</artifactId>
            <version>1.1.22</version>
        </dependency>

        <dependency>
            <groupId>io.shardingsphere</groupId>
            <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
            <version>3.1.0.M1</version>
        </dependency>

        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
            <version>3.1.2</version>
        </dependency>

        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-generator</artifactId>
            <version>3.1.2</version>
        </dependency>

application.properties configuration

# 服务端口
server.port=8080

# mysql-plus 配置
mybatis-plus.mapper-locations=classpath:/mapper/*.xml
mybatis-plus.type-aliases-package=com.stwen.shardingjdbc.entity

spring.main.allow-bean-definition-overriding=true

# sharding-jdbc 配置主从
sharding.jdbc.dataSource.names=master,slave1,slave2

# sharding-jdbc 主数据库
sharding.jdbc.dataSource.master.type=com.alibaba.druid.pool.DruidDataSource
sharding.jdbc.dataSource.master.driverClassName=com.mysql.jdbc.Driver
sharding.jdbc.dataSource.master.url=jdbc:mysql://192.168.239.128:3307/test?useUnicode=true&characterEncoding=utf8&tinyInt1isBit=false&useSSL=false&serverTimezone=GMT
sharding.jdbc.dataSource.master.username=root
sharding.jdbc.dataSource.master.password=123456
sharding.jdbc.dataSource.master.maxPoolSize=20

# sharding-jdbc 从数据库一
sharding.jdbc.dataSource.slave1.type=com.alibaba.druid.pool.DruidDataSource
sharding.jdbc.dataSource.slave1.driverClassName=com.mysql.jdbc.Driver
sharding.jdbc.dataSource.slave1.url=jdbc:mysql://192.168.239.128:3308/test?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT
sharding.jdbc.dataSource.slave1.username=root
sharding.jdbc.dataSource.slave1.password=123456
sharding.jdbc.dataSource.slave1.maxPoolSize=20

# sharding-jdbc 从数据库二
sharding.jdbc.dataSource.slave2.type=com.alibaba.druid.pool.DruidDataSource
sharding.jdbc.dataSource.slave2.driverClassName=com.mysql.jdbc.Driver
sharding.jdbc.dataSource.slave2.url=jdbc:mysql://192.168.239.128:3309/test?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT
sharding.jdbc.dataSource.slave2.username=root
sharding.jdbc.dataSource.slave2.password=123456
sharding.jdbc.dataSource.slave2.maxPoolSize=20

# 配置从库选择策略，提供轮询与随机，这里选择用轮询，random-随机
sharding.jdbc.config.masterslave.load-balance-algorithm-type=round_robin
# 配置主从读写分离
sharding.jdbc.config.masterslave.name=master-slave 
sharding.jdbc.config.masterslave.master-data-source-name=master
sharding.jdbc.config.masterslave.slave-data-source-names=slave1,slave2
# 开启SQL显示，默认值: false，注意：仅配置读写分离时不会打印日志
sharding.jdbc.props.sql.show=true

Configuration parameter description:

• mybatis-plus.mapper-locations : the path of the XXXmapper.xml mapping file
• mybatis-plus.type-aliases-package : entity class package name
• sharding.jdbc.dataSource.names : The configuration is the name of the database, which is the name of the multiple data sources built above
• sharding.jdbc.dataSource : Configure multiple data sources

The other detailed parameters will not be explained one by one, just look at it and understand.

Among them, spring.main.allow-bean-definition-overriding=true is added to prevent the following errors from being reported:

In the Springboot startup class, add configuration, scan the dao interface package of mybatis, as follows

test

Write test class, add two interfaces: add new, query list

@RestController
@RequestMapping("/user")
public class UserController {

    @Autowired
    private IUserService userService;

    @PutMapping("/save")
    public Object save() {
        User user = new User();
        user.setName("新增");
        return userService.save(user);
    }
    
    @GetMapping("/list")
    public Object list() {
        // 强制路由主库
        //HintManager.getInstance().setMasterRouteOnly();
        return userService.list();
    }
    
}

Start the application, access the interface: localhost:8080/user/save, you can view the data, and find that a new piece of data has been added

Visit the list query interface again: localhost:8080/user/list

Enable log (not required)

(Not necessary) Turn on the log of sql statement. It is not recommended to turn it on in the production environment. This is just to verify whether the effect of reading and writing separation is achieved.

View the log directory and open the log of sql statements:

mysql>  show variables like '%general_log%';
mysql>  set global general_log=on;

Note:

• Connect to the three databases configured above, execute the above command, and turn on sql logging (no need to restart to take effect).
• When Mysql is restarted, the above log configuration will be invalid.

After the three databases are set up, visit the two test interfaces again, and then check the sql log:

As follows, in the mysql-master container, check the sql log, you can see that the inserted sql is executed in the main database

As follows, check the sql log of mysql-slave2, and read list data is executed from the database, indicating that the read-write separation we configured is successful.

Master-slave synchronization delay

　　It often appears in the read-write separation architecture, that is , how to solve the problem of read delay ?

　　Just insert a piece of data, and then read it right away, may it not be read at this time? In the final analysis, it is because the data is copied to the slave node after the master node is written. The reason why the data cannot be read is that the copying time is relatively long, that is to say, the data has not been copied to the slave node, you have to read from the node I can’t read it. The master-slave replication of mysql5.7 is multi-threaded , which means that the speed will become faster, but it may not be guaranteed to be read 100% immediately . We can solve this problem in two ways:

　　(1) Compromise at the business level, whether it will be read immediately after the operation is completed

　　(2) For those that will be read out immediately after the operation and cannot be compromised in business, we can go directly to the main library for this type of reading . Of course, Sharding-JDBC also considers the existence of this problem, so we provide us with a The function allows the user to specify whether to use the main library for reading. Use the following method to set up before reading:

    @GetMapping("/list")
    public Object list() {
        // 强制路由主库
        HintManager.getInstance().setMasterRouteOnly();
        return userService.list();
    }

Check the sql log of the main library configuration, you will see the following printout, indicating the main library that the read operation is forced to go:

SELECT  id,name  FROM t_user

●The strongest Tomcat8 performance optimization in history

● Why can Alibaba resist 10 billion in 90 seconds? --The evolution of server-side high-concurrency distributed architecture

● B2B e-commerce platform--ChinaPay UnionPay electronic payment function

● Learn Zookeeper distributed lock, let interviewers look at you with admiration

● SpringCloud e-commerce spike microservice-Redisson distributed lock solution

Check out more good articles, enter the official account--please me--excellent in the past

A deep and soulful public account 0.0