springBoot+myBatis configures read-write separation based on mysql

1. What is database read-write separation?

Database read-write separation is to divide the database into a master-slave library, and the master library is used to write data and maintain the data by adding, deleting, and modifying it. The slave library synchronizes data from the main library through a synchronization mechanism, that is, a full mirror backup of the main library. Read-write separation is generally to configure a master library with multiple slave libraries, which is a common database architecture design.

2. What problem does database reading and writing separation solve?

Database read and write separation is mainly to solve the bottleneck of data reading performance in the business. Because most systems generally read more and write less, at this time, the read operation will first become the bottleneck of the database service, and indirectly cause problems in the database write. Generally, the Internet system adopts a single-database design architecture at the beginning, that is, a common database for reading and writing. However, with the expansion of business and the increase of system users, reading and writing of a single database will be stuck, and even business operations will fail. . At this time, the design idea of ​​reading and writing separation can meet the needs of the system to a certain extent. Proper use of the database architecture with read-write separation can linearly improve the performance bottleneck of database read operations, and at the same time solve the impact of read-write lock conflicts on write operations and improve the performance of write operations.

Note: Read-write separation is only a design idea to temporarily solve the database performance conflict due to the increase in business and user volume of the system to a certain extent, but it cannot completely solve this problem. When the system size is large enough, read-write separation will not be able to cope with system performance problems. At this time, other design ideas are needed to solve it, such as database sub-library, partition, and table sub-segmentation.

3. Implementation of reading and writing separation code (springboot+mybatis+mysql)

1. First, you need to configure the master-slave library of myslql. For this configuration, please refer to this article written before.

2. Code implementation
1. Create a new springboot project in idea

2. Add the following dependency injection configuration information to pom.xml

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- 使用TkMybatis可以无xml文件实现数据库操作,只需要继承tkMybatis的Mapper接口即可-->
<dependency>
    <groupId>tk.mybatis</groupId>
    <artifactId>mapper-spring-boot-starter</artifactId>
    <version>1.1.4</version>
</dependency>
<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>druid</artifactId>
    <version>1.0.9</version>
</dependency>
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>

3. Create a new application.yml file in the src/main/resources directory and configure the following information

server:
  port: 8082
  servlet:
    context-path: /dxfl
spring:
  datasource:
    #读库数目
    maxReadCount: 1
    type-aliases-package: com.teamo.dxfl.mapper
    mapper-locations: classpath:/mapper/*.xml
    config-location: classpath:/mybatis-config.xml
    write:
      url: jdbc:mysql://localhost:3306/test?useUnicode=true&characterEncoding=utf-8&useSSL=true
      username: root
      password:
      driver-class-name: com.mysql.jdbc.Driver
      initialSize: 2           #初始化大小
      maxWait: 6000        #获取连接时最大等待时间，单位毫秒。
      min-idle: 5            # 数据库连接池的最小维持连接数
      maxActive: 20         # 最大的连接数
      initial-size: 5          # 初始化提供的连接数
      max-wait-millis: 200    # 等待连接获取的最大超时时间
    read1:
      url: jdbc:mysql://localhost:3307/test?useUnicode=true&characterEncoding=utf-8&useSSL=true
      username: root
      password:
      driver-class-name: com.mysql.jdbc.Driver
      initialSize: 2          #初始化大小
      maxWait: 6000       #获取连接时最大等待时间，单位毫秒。
      min-idle: 5           # 数据库连接池的最小维持连接数
      maxActive: 20        # 最大的连接数
      initial-size: 5         # 初始化提供的连接数
      max-wait-millis: 200   # 等待连接获取的最大超时时间

4. Write the configuration class of the data source (under the config directory)
DataSourceConfig.java

@Configuration
public class DataSourceConfig {
    @Value("${spring.datasource.type-aliases-package}")
    private String typeAliasesPackage;

    @Value("${spring.datasource.mapper-locations}")
    private String mapperLocation;

    @Value("${spring.datasource.config-location}")
    private String configLocation;
    /**
     * 写数据源
     * @Primary 标志这个 Bean 如果在多个同类 Bean 候选时，该 Bean 优先被考虑。
     * 多数据源配置的时候注意，必须要有一个主数据源，用 @Primary 标志该 Bean
     */
    @Primary
    @Bean
    @ConfigurationProperties(prefix = "spring.datasource.write")
    public DataSource writeDataSource() {
        return new DruidDataSource();
    }

    /**
     * 读数据源
     */
    @Bean
    @ConfigurationProperties(prefix = "spring.datasource.read1")
    public DataSource readDataSource1() {
        return new DruidDataSource();
    }

    /**
     * 多数据源需要自己设置sqlSessionFactory
     */
    @Bean
    public SqlSessionFactory sqlSessionFactory() throws Exception {
        SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
        bean.setDataSource(routingDataSource());
        ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
        // 实体类对应的位置
        bean.setTypeAliasesPackage(typeAliasesPackage);
        // mybatis的XML的配置
        bean.setMapperLocations(resolver.getResources(mapperLocation));
        bean.setConfigLocation(resolver.getResource(configLocation));
        return bean.getObject();
    }

    /**
     * 设置数据源路由，通过该类中的determineCurrentLookupKey决定使用哪个数据源
     */
    @Bean
    public AbstractRoutingDataSource routingDataSource() {
        RoutingDataSourceConfig proxy = new RoutingDataSourceConfig();

        DataSource writeDataSource = writeDataSource();

        //设置数据源Map对象
        Map<Object, Object> dataSource = new HashMap<Object, Object>(2);
        dataSource.put(DataBaseTypeEnum.WRITE.getCode(), writeDataSource);

        //如果配置了多个读数据源，就一次添加到datasource对象中
        dataSource.put(DataBaseTypeEnum.READ.getCode()+"1", readDataSource1());

        //写数据源设置为默认数据源
        proxy.setDefaultTargetDataSource(writeDataSource);
        proxy.setTargetDataSources(dataSource);
        return proxy;
    }

    /**
     * 设置事务，事务需要知道当前使用的是哪个数据源才能进行事务处理(配置mybatis时候用到)
    */
     @Bean
     public DataSourceTransactionManager dataSourceTransactionManager() {
        return new DataSourceTransactionManager(routingDataSource());
     }
}

5. Write a configuration class for data source routing (under the config directory), which inherits AbstractRoutingDataSource and rewrites the determineCurrentLookupKey() method, which implements the logic of data source selection. At the same time, this class holds a ThreadLocal object, which is used to store whether the data source type used by the current thread is read or write. You can write a certain algorithm by yourself to realize the load balancing of the reading library (adding to the reading library and configuring more than one). Here, it is simple to choose which reading database data source to use by obtaining random numbers.
DataSourceRouteConfig.java

public class RoutingDataSourceConfig extends AbstractRoutingDataSource {
    //使用ThreadLocal对象保存当前线程是否处于读模式
    private static ThreadLocal<String> DataBaseMap= new ThreadLocal<>();

    @Value("${spring.datasource.maxReadCount}")
    private int maxReadCount;

    private final Logger log = LoggerFactory.getLogger(this.getClass());

    @Override
    protected Object determineCurrentLookupKey() {
        String typeKey = getDataBaseType();
        if (typeKey == DataBaseTypeEnum.WRITE.getCode()) {
            log.info("使用了写库");
            return typeKey;
        }
        //使用随机数决定使用哪个读库
        int index = (int) Math.floor(Math.random() * (maxReadCount - 1 + 1)) + 1;;
        log.info("使用了读库{}", index);
        return DataBaseTypeEnum.READ.getCode() + index;
    }

    public static void setDataBaseType(String dataBaseType) {
        DataBaseMap.set(dataBaseType);
    }

    public static String getDataBaseType() {
        return DataBaseMap.get() == null ? DataBaseTypeEnum.WRITE.getCode() : DataBaseMap.get();
    }

    public static void clearDataBaseType() {
        DataBaseMap.remove();
    }
}

6. We need to write an Annotation class, which is specially used to mark which methods of the Service class use the read data source and which methods use the write data source. (In addition to this method, you can also configure the matching method name in the aop class, such as wildcard save*, update*, delete* methods to match the write data source, other methods are read data sources, only considered here Annotation annotation method).

ReadDB.java
@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
public @interface ReadDB {
    //String value() default "";
}

7. Write the aop class. Before requesting to open the transaction, use spring's aop (aspect-oriented programming) to first judge whether the ReadDB annotation is on the Service method. If the ReadDB annotation is on the method, select the read data source. This class implements the Ordered interface and rewrites the getOrder() method. This interface notifies Spring to call the class to execute the method order. The smaller the value returned by the getOrder() method, the higher the priority. It should be noted here that the value returned by the getOrder() method must be less than the value set for @EnableTransactionManagement(order= ) in the project startup class , so as to ensure that the operation of selecting a data source is executed before starting the transaction.
DxflApplication.java

@SpringBootApplication
@EnableTransactionManagement(order = 5)
public class DxflApplication {
    public static void main(String[] args) {
        SpringApplication.run(DxflApplication.class, args);
    }
}

ReadDBAspect.java

@Aspect
@Component
public class ReadDBAspect implements Ordered {
    private static final Logger log= LoggerFactory.getLogger(ReadDBAspect.class);
    @Around("@annotation(readDB)")
    public Object setRead(ProceedingJoinPoint joinPoint, ReadDB readDB) throws Throwable {
        try{
            //设置读数据源
            RoutingDataSourceConfig.setDataBaseType(DataBaseTypeEnum.READ.getCode());
            return joinPoint.proceed();
        }finally {
            log.info("清除dataSource类型选中值："+DataBaseTypeEnum.READ.getData());
            RoutingDataSourceConfig.clearDataBaseType();
        }
    }
    @Override
    public int getOrder() {
        return 0;
    }
}

8. Write the Service method, add the @ReadDB annotation to the method of reading data, and mark the method as the method of reading the library.
UserServiceImpl.java

@Service
public class UserviceServiceImpl implements UserService {
    @Autowired
    private UserMapper userMapper;

    @Override
    @ReadDB
    public List<User> getAllUser() {
        return userMapper.selectAll();
    }

    @Override
    @Transactional(rollbackFor = RuntimeException.class)
    public boolean save(User user){
        if(userMapper.insert(user)>0){
            return true;
        }else{
            return false;
        }
        //throw new RuntimeException("测试事务");
    }
}

9. Write the Controller class to call the Service method to implement the test of the read and write methods.
UserController.java

@RestController
@RequestMapping("/user")
public class UserController{
    @Autowired
    private UserService userService;

    @GetMapping("getAllUser")
    public List<User> getAllUser() {
        List resList = userService.getAllUser();
        return resList;
    }

    @PostMapping("save")
    public Result save(@RequestBody User user){
        Result result = new Result();
        if(null != user){
            try{
                userService.save(user);
                result.setCode(ResultEnum.SUCCESS.getCode());
                result.setMsg("保存用户成功！");
            }catch(Exception e) {
                e.printStackTrace();
                result.setCode(ResultEnum.SUCCESS.getCode());
                result.setMsg("保存用户失败，原因："+ e.getMessage() +"！");
            }
        }else{
            result.setCode(ResultEnum.SUCCESS.getCode());
            result.setMsg("保存用户失败,原因：用户数据为空！");
        }
        return result;
    }
}

10. Test
After successfully starting the service, test the library reading: enter the address in the browser, http://localhost:8089/dxfl/user/getAllUser, the display is as follows: Check the
background printing: it shows that the library is used
to test the library test and
write the library Form submission is required, and a test tool is required at this time, and ApiPost is used here. Open the test tool, enter the interface for saving the user in the test column, select application/json as the submission method, and then click the send button, and a successful return message will be displayed with a response, and a prompt to save the user means that the writing test is successful.
The back-end log print information is: the write library is used, and the table name test write library is successful.
In order to successfully verify whether the read-write separation is really successful, you can first turn off the function of the total slave library (stop slave) when testing the write library, and then turn on the function of the master-slave library (start slave) after ensuring that the specified write library is written .