SpringBoot project elegantly implements read-write separation

From data to large model application, on November 25th, Hangzhou Yuanchuang Conference was held to share development tips.

1. Introduction to reading and writing separation

When using Spring Boot to develop database applications, separation of read and write is a common optimization strategy. Read-write separation allocates read operations and write operations to different database instances to improve system throughput and performance.

The implementation of read-write separation is mainly realized through the dynamic data source function. Dynamic data source is a mechanism that dynamically switches database connections at runtime. It allows applications to select different data sources based on different conditions or configurations to achieve more flexible and scalable database access.

2. Achieve reading and writing separation - basics

1. Configure the connection information of the master database and slave database

  
  
  
  
  
   
   
   
   
   
   
   
   
   # 主库配置spring.datasource.master.jdbc-url=jdbc:mysql://ip:port/master?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=falsespring.datasource.master.username=masterspring.datasource.master.password=123456spring.datasource.master.driver-class-name=com.mysql.jdbc.Driver
# 从库配置spring.datasource.slave.jdbc-url=jdbc:mysql://ip:port/slave?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=falsespring.datasource.slave.username=slavespring.datasource.slave.password=123456spring.datasource.slave.driver-class-name=com.mysql.jdbc.Driver

2. Create data source configuration classes for the master database and slave database

The creation of different data sources can be completed through different condition restrictions and configuration file prefixes, not only master-slave but also multiple different databases

Main database data source configuration

  
  
  
  
  
   
   
   
   
   
   
   
   
   @Configuration@ConditionalOnProperty("spring.datasource.master.jdbc-url")public class MasterDataSourceConfiguration {    @Bean("masterDataSource")    @ConfigurationProperties(prefix = "spring.datasource.master")    public DataSource masterDataSource() {        return DataSourceBuilder.create().build();    }}

Configuration from database data source

  
  
  
  
  
   
   
   
   
   
   
   
   
   @Configuration@ConditionalOnProperty("spring.datasource.slave.jdbc-url")public class SlaveDataSourceConfiguration {    @Bean("slaveDataSource")    @ConfigurationProperties(prefix = "spring.datasource.slave")    public DataSource slaveDataSource() {        return DataSourceBuilder.create().build();    }}

3. Create master-slave data source enumeration

  
  
  
  
  
   
   
   
   
   
   
   
   
   public enum DataSourceTypeEnum {    /**     * 主库     */    MASTER,
    /**     * 从库     */    SLAVE,   ;  }

4. Create a dynamic routing data source

A switch is made here to control the opening and closing of read-write separation, and all operations can be switched to the main library. Then return different data source type enumerations based on the data source type in the context

  
  
  
  
  
   
   
   
   
   
   
   
   
   @Slf4jpublic class DynamicRoutingDataSource extends AbstractRoutingDataSource {
    @Value("${DB_RW_SEPARATE_SWITCH:false}")    private boolean dbRwSeparateSwitch;    @Override    protected Object determineCurrentLookupKey() {        if(dbRwSeparateSwitch && DataSourceTypeEnum.SLAVE.equals(DataSourceContextHolder.getDataSourceType())) {            log.info("DynamicRoutingDataSource 切换数据源到从库");            return DataSourceTypeEnum.SLAVE;        }        log.info("DynamicRoutingDataSource 切换数据源到主库");        // 根据需要指定当前使用的数据源，这里可以使用ThreadLocal或其他方式来决定使用主库还是从库        return DataSourceTypeEnum.MASTER;    }}

5. Create a dynamic data source configuration class

Add the data sources of the master database and slave database to the dynamic data source, and create a data source map through enumeration, so that the data source can be switched through the enumeration returned by the above route.

  
  
  
  
  
   
   
   
   
   
   
   
   
   @Configuration@ConditionalOnProperty("spring.datasource.master.jdbc-url")public class DynamicDataSourceConfiguration {    @Bean("dataSource")    @Primary    public DataSource dynamicDataSource(DataSource masterDataSource, DataSource slaveDataSource) {        Map<Object, Object> targetDataSources = new HashMap<>();        targetDataSources.put(DataSourceTypeEnum.MASTER, masterDataSource);        targetDataSources.put(DataSourceTypeEnum.SLAVE, slaveDataSource);
        DynamicRoutingDataSource dynamicDataSource = new DynamicRoutingDataSource();        dynamicDataSource.setTargetDataSources(targetDataSources);        dynamicDataSource.setDefaultTargetDataSource(masterDataSource);        return dynamicDataSource;    }}

6. Create the DatasourceContextHolder class and use ThreadLocal to store the data source type of the current thread.

Note that there is a potential risk here that when creating a new thread, the data in ThreadLocal cannot be read correctly. If it involves starting a new thread, you can use TransmittableThreadLocal to synchronize the data of the parent and child threads. Git address: https://github . com/alibaba/transmittable-thread-local

  
  
  
  
  
   
   
   
   
   
   
   
   
   public class DataSourceContextHolder {    private static final ThreadLocal<DataSourceTypeEnum> contextHolder = new ThreadLocal<>();
    public static void setDataSourceType(DataSourceTypeEnum dataSourceType) {        contextHolder.set(dataSourceType);    }
    public static DataSourceTypeEnum getDataSourceType() {        return contextHolder.get();    }
    public static void clearDataSourceType() {        contextHolder.remove();    }}

7. Create custom annotations to mark master and slave data sources

  
  
  
  
  
   
   
   
   
   
   
   
   
   @Retention(RetentionPolicy.RUNTIME)@Target(ElementType.METHOD)public @interface MasterDataSource {}@Retention(RetentionPolicy.RUNTIME)@Target(ElementType.METHOD)public @interface SlaveDataSource {}

8. Create aspect classes, intercept database operations, and switch data source parameters according to annotation settings

  
  
  
  
  
   
   
   
   
   
   
   
   
   @Aspect@Componentpublic class DataSourceAspect {
    @Before("@annotation(xxx.MasterDataSource)")    public void setMasterDataSource(JoinPoint joinPoint) {        DataSourceContextHolder.setDataSourceType(DataSourceTypeEnum.MASTER);    }
    @Before("@annotation(xxx.SlaveDataSource)")    public void setSlaveDataSource(JoinPoint joinPoint) {        DataSourceContextHolder.setDataSourceType(DataSourceTypeEnum.SLAVE);    }
    @After("@annotation(xxx.MasterDataSource) || @annotation(xxx.SlaveDataSource)")    public void clearDataSource(JoinPoint joinPoint) {        DataSourceContextHolder.clearDataSourceType();    }}

9. Use custom annotation tags on the Service layer method to query the data source

  
  
  
  
  
   
   
   
   
   
   
   
   
   @Servicepublic class TestService {    @Autowired    private TestDao testDao;
    @SlaveDataSource    public Test test() {        return testDao.queryByPrimaryKey(11L);    }}

10. Exclude data source automatic configuration class

Failure to exclude automatic configuration classes will cause problems with initializing multiple dataSource objects.

  
  
  
  
  
   
   
   
   
   
   
   
   
   SpringBootApplication(exclude = {DataSourceAutoConfiguration.class})

3. Achieve reading and writing separation-advanced

1. Use connection pool, take Hikari as an example

Modify the link configuration and add the link pool related configuration.

  
  
  
  
  
   
   
   
   
   
   
   
   
   # 主库配置spring.datasource.master.jdbc-url=jdbc:mysql://ip:port/master?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=falsespring.datasource.master.username=masterspring.datasource.master.password=123456spring.datasource.master.driver-class-name=com.mysql.jdbc.Driverspring.datasource.master.type=com.zaxxer.hikari.HikariDataSourcespring.datasource.master.hikari.name=masterspring.datasource.master.hikari.minimum-idle=5spring.datasource.master.hikari.idle-timeout=30spring.datasource.master.hikari.maximum-pool-size=10spring.datasource.master.hikari.auto-commit=truespring.datasource.master.hikari.pool-name=DatebookHikariCPspring.datasource.master.hikari.max-lifetime=1800000spring.datasource.master.hikari.connection-timeout=30000spring.datasource.master.hikari.connection-test-query=SELECT 1
# 从库配置spring.datasource.slave.jdbc-url=jdbc:mysql://ip:port/slave?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=falsespring.datasource.slave.username=rootspring.datasource.slave.password=123456spring.datasource.slave.driver-class-name=com.mysql.jdbc.Driverspring.datasource.slave.type=com.zaxxer.hikari.HikariDataSourcespring.datasource.slave.hikari.name=masterspring.datasource.slave.hikari.minimum-idle=5spring.datasource.slave.hikari.idle-timeout=30spring.datasource.slave.hikari.maximum-pool-size=10spring.datasource.slave.hikari.auto-commit=truespring.datasource.slave.hikari.pool-name=DatebookHikariCPspring.datasource.slave.hikari.max-lifetime=1800000spring.datasource.slave.hikari.connection-timeout=30000spring.datasource.slave.hikari.connection-test-query=SELECT 1

2. Integrate mybatis and force switch to the main library when writing

No configuration is required, and the read-write separation function can be used by integrating mybatis normally.

You can use the interceptor of mybatis to force switching to the main library during write operations.

  
  
  
  
  
   
   
   
   
   
   
   
   
   @Intercepts({        @Signature(type = Executor.class, method = "update", args = {MappedStatement.class, Object.class}),})@Componentpublic class WriteInterceptor implements Interceptor {    @Override    public Object intercept(Invocation invocation) throws Throwable {        // 获取 SQL 类型        DataSourceTypeEnum dataSourceType = DataSourceContextHolder.getDataSourceType();        if(DataSourceTypeEnum.SLAVE.equals(dataSourceType)) {            DataSourceContextHolder.setDataSourceType(DataSourceTypeEnum.MASTER);        }        try {            // 执行 SQL            return invocation.proceed();        } finally {            // 恢复数据源  考虑到写入后可能会反查，后续都走主库            // DataSourceContextHolder.setDataSourceType(dataSourceType);        }    }}

-end-

This article is shared from the WeChat public account - JD Cloud Developers (JDT_Developers).
If there is any infringement, please contact [email protected] for deletion.
This article participates in the " OSC Source Creation Plan ". You who are reading are welcome to join and share together.