Based on custom annotations and Aop dynamic data source configuration
In actual projects, strategies such as sub-database sub-table or read-write separation are often designed because of the need to enhance database concurrency. Every time a new technology is introduced in an old project, it will bring a series of problems. Our purpose is to solve it. The problem is to reconstruct the system with a way of thinking, and find fun from it. Corresponding to the problems caused by the introduction of custom annotations and Aop dynamic data source configuration technology, I will introduce it at the end of the article. I also hope that God will give correct guidance. The requirement is: there is an old XXX system, we develop a PC-side program on the basis of this old system for cashier; the other party provides their database documents and docking personnel, but they do not give the old system code, we can only understand through communication With their old system design ideas, they took 10,000 fuckers to write code; we belong to secondary development and need to develop our own business database based on the database of the old system, and here we have designed two databases ( One is the database of the old system, and the other is the database of the cashier system. Before the project, I could think of custom annotations and Aop dynamic data source configuration to achieve it, but there are pits. I will propose the pits below; now let's start with the configuration. (This article is based on dynamic data source switching integrated under the SSM framework):
1. Configure pom.xml, using Alibaba data source package and Mysql 5.1.30 driver
<!-- 阿里巴巴数据源包 -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid</artifactId>
<version>1.0.2</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.30</version>
</dependency>2. The core configuration of spring-dispatcher.xml is as follows:
<context:property-placeholder location="classpath:config.properties" />
<!-- 阿里 druid数据库连接池 -->
<bean id="masterDataSource" class="com.alibaba.druid.pool.DruidDataSource"
destroy-method="close">
<!-- 数据库基本信息配置 -->
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<property name="driverClassName" value="${jdbc.driverClassName}" />
<property name="filters" value="${jdbc.filters}" />
<!-- 最大并发连接数 -->
<property name="maxActive" value="${jdbc.maxActive}" />
<!-- 初始化连接数量 -->
<property name="initialSize" value="${jdbc.initialSize}" />
<!-- 配置获取连接等待超时的时间 -->
<property name="maxWait" value="${jdbc.maxWait}" />
<!-- 最小空闲连接数 -->
<property name="minIdle" value="${jdbc.minIdle}" />
<!-- 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒 -->
<property name="timeBetweenEvictionRunsMillis" value="${jdbc.timeBetweenEvictionRunsMillis}" />
<!-- 配置一个连接在池中最小生存的时间，单位是毫秒 -->
<property name="minEvictableIdleTimeMillis" value="${jdbc.minEvictableIdleTimeMillis}" />
<property name="validationQuery" value="${jdbc.validationQuery}" />
<property name="testWhileIdle" value="${jdbc.testWhileIdle}" />
<property name="testOnBorrow" value="${jdbc.testOnBorrow}" />
<property name="testOnReturn" value="${jdbc.testOnReturn}" />
<property name="maxOpenPreparedStatements" value="${jdbc.maxOpenPreparedStatements}" />
<!-- 打开removeAbandoned功能 -->
<property name="removeAbandoned" value="${jdbc.removeAbandoned}" />
<!-- 1800秒，也就是30分钟 -->
<property name="removeAbandonedTimeout" value="${jdbc.removeAbandonedTimeout}" />
<!-- 关闭abanded连接时输出错误日志 -->
<property name="logAbandoned" value="${jdbc.logAbandoned}" />
</bean>
<!-- 阿里 druid数据库连接池 -->
<bean id="slaveDataSource" class="com.alibaba.druid.pool.DruidDataSource"
destroy-method="close">
<!-- 数据库基本信息配置 -->
<property name="url" value="${jdbc.slave.url}" />
<property name="username" value="${jdbc.slave.username}" />
<property name="password" value="${jdbc.slave.password}" />
<property name="driverClassName" value="${jdbc.slave.driverClassName}" />
<property name="filters" value="${jdbc.filters}" />
<!-- 最大并发连接数 -->
<property name="maxActive" value="${jdbc.maxActive}" />
<!-- 初始化连接数量 -->
<property name="initialSize" value="${jdbc.initialSize}" />
<!-- 配置获取连接等待超时的时间 -->
<property name="maxWait" value="${jdbc.maxWait}" />
<!-- 最小空闲连接数 -->
<property name="minIdle" value="${jdbc.minIdle}" />
<!-- 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒 -->
<property name="timeBetweenEvictionRunsMillis" value="${jdbc.timeBetweenEvictionRunsMillis}" />
<!-- 配置一个连接在池中最小生存的时间，单位是毫秒 -->
<property name="minEvictableIdleTimeMillis" value="${jdbc.minEvictableIdleTimeMillis}" />
<property name="validationQuery" value="${jdbc.validationQuery}" />
<property name="testWhileIdle" value="${jdbc.testWhileIdle}" />
<property name="testOnBorrow" value="${jdbc.testOnBorrow}" />
<property name="testOnReturn" value="${jdbc.testOnReturn}" />
<property name="maxOpenPreparedStatements" value="${jdbc.maxOpenPreparedStatements}" />
<!-- 打开removeAbandoned功能 -->
<property name="removeAbandoned" value="${jdbc.removeAbandoned}" />
<!-- 1800秒，也就是30分钟 -->
<property name="removeAbandonedTimeout" value="${jdbc.removeAbandonedTimeout}" />
<!-- 关闭abanded连接时输出错误日志 -->
<property name="logAbandoned" value="${jdbc.logAbandoned}" />
</bean>
<bean id="dynamicDataSource" class="cn.edu.his.pay.dynamic.datasource.DynamicDataSource">
<property name="targetDataSources">
<map key-type="java.lang.String">
<!-- 指定lookupKey和与之对应的数据源 -->
<entry key="MASTER" value-ref="masterDataSource"></entry>
<entry key="SLAVE" value-ref="slaveDataSource"></entry>
</map>
</property>
<!-- 这里可以指定默认的数据源 -->
<property name="defaultTargetDataSource" ref="masterDataSource" />
</bean>
<!-- mybatis文件配置，扫描所有mapper*xml.文件 -->
<bean id="sessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
<property name="dataSource" ref="dynamicDataSource" />
<property name="typeAliasesPackage" value="cn.edu.his.pay.model.entity" />
<property name="mapperLocations" value="classpath:mybatis/xml/*Mapper.xml" />
<property name="plugins">
<array>
<bean class="com.github.pagehelper.PageHelper">
<property name="properties">
<value>
dialect=mysql
reasonable=true
</value>
</property>
</bean>
</array>
</property>
</bean>
<!-- spring与mybatis整合配置，扫描所有mapper -->
<bean class="org.mybatis.spring.mapper.MapperScannerConfigurer">
<property name="basePackage" value="cn.edu.his.pay.mapper" />
<property name="sqlSessionFactoryBeanName" value="sessionFactory" />
</bean>
<!-- 对数据源进行事务管理 -->
<bean id="transactionManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dynamicDataSource" />
</bean>
<!-- 配置哪些方法要加入事务控制 -->
<tx:advice id="txAdvice" transaction-manager="transactionManager">
<tx:attributes>
<!-- 让所有的方法都加入事务管理，为了提高效率，可以把一些查询之类的方法设置为只读的事务 -->
<tx:method name="*" propagation="REQUIRED" read-only="true" />
<!-- 以下方法都是可能设计修改的方法，就无法设置为只读 -->
<tx:method name="add*" propagation="REQUIRED" />
<tx:method name="insert*" propagation="REQUIRED" />
<tx:method name="del*" propagation="REQUIRED" />
<tx:method name="update*" propagation="REQUIRED" />
<tx:method name="save*" propagation="REQUIRED" />
<tx:method name="clear*" propagation="REQUIRED" />
<tx:method name="handle*" propagation="REQUIRED" />
</tx:attributes>
</tx:advice>
<!-- 配置AOP，Spring是通过AOP来进行事务管理的 -->
<aop:config>
<!-- 设置pointCut表示哪些方法要加入事务处理 -->
<!-- 以下的事务是声明在DAO中，但是通常都会在Service来处理多个业务对象逻辑的关系，注入删除，更新等，此时如果在执行了一个步骤之后抛出异常
就会导致数据不完整，所以事务不应该在DAO层处理，而应该在service，这也就是Spring所提供的一个非常方便的工具，声明式事务 -->
<aop:pointcut id="allMethods" expression="(execution(* cn.edu.his.pay.service.*.*(..)))" />
<!-- 通过advisor来确定具体要加入事务控制的方法 -->
<aop:advisor advice-ref="txAdvice" pointcut-ref="allMethods" />
</aop:config>3. The config.properties configuration file that spring-dispatcher.xml depends on is as follows:
# =====================数据源切换数据master和slave数据库=====================
# master 也是默认的数据源（默认为旧系统的：原因是他们的表比较多）
jdbc.url=jdbc:mysql://127.0.0.1:3306/his?useUnicode=true&characterEncoding=utf8
jdbc.driverClassName=com.mysql.jdbc.Driver
jdbc.username=root
jdbc.password=root2
# slave 需要切换的数据源（slave，原因是我们的表比较少）
jdbc.slave.url=jdbc:mysql://127.0.0.1:3306/his_pay?useUnicode=true&characterEncoding=utf8
jdbc.slave.driverClassName=com.mysql.jdbc.Driver
jdbc.slave.username=root
jdbc.slave.password=root
# =====================数据源切换数据master和slave数据库=====================
jdbc.filters=stat
jdbc.maxActive=20
jdbc.initialSize=1
jdbc.maxWait=60000
jdbc.minIdle=10
jdbc.maxIdle=15
jdbc.timeBetweenEvictionRunsMillis=60000
jdbc.minEvictableIdleTimeMillis=300000
jdbc.validationQuery=SELECT 'x'
jdbc.testWhileIdle=true
jdbc.testOnBorrow=false
jdbc.testOnReturn=false
jdbc.maxOpenPreparedStatements=20
jdbc.removeAbandoned=true
jdbc.removeAbandonedTimeout=1800
jdbc.logAbandoned=true4. There are the following classes under the dynamic.datasource package in the same directory as the controller package:
DataSource.java (custom annotation), DataSourceAspect.java (Aop aspect), DataSourceType.java (enumeration: used to specify the data source name), DynamicDataSource.java, DynamicDataSourceHolder.java.
5. DataSource.java is as follows:
package cn.edu.his.pay.dynamic.datasource;
import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/*
@Target(ElementType.TYPE) //接口、类、枚举、注解
@Target(ElementType.FIELD) //字段、枚举的常量
@Target(ElementType.METHOD) //方法
@Target(ElementType.PARAMETER) //方法参数
@Target(ElementType.CONSTRUCTOR) //构造函数
@Target(ElementType.LOCAL_VARIABLE)//局部变量
@Target(ElementType.ANNOTATION_TYPE)//注解
@Target(ElementType.PACKAGE) ///包
@Retention(RetentionPolicy.SOURCE) //注解仅存在于源码中，在class字节码文件中不包含
@Retention(RetentionPolicy.CLASS) //默认的保留策略，注解会在class字节码文件中存在，但运行时无法获得，
@Retention(RetentionPolicy.RUNTIME)//注解会在class字节码文件中存在，在运行时可以通过反射获取到
*/
/**
* @author 93287
*
*/
@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface DataSource {
DataSourceType value();
}6. DataSourceAspect.java is as follows:
package cn.edu.his.pay.dynamic.datasource;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.annotation.After;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Before;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
import cn.edu.his.pay.common.log.Logger;
@Aspect
@Order(-1)
// 保证该AOP在@Transactional之前执行
@Component
public class DataSourceAspect {
private static final Logger LOG = new Logger(DataSourceAspect.class);
@Before("@annotation(ds)")
public void changeDataSource(JoinPoint point, DataSource ds) throws Throwable {
LOG.debug("=======================SET START=======================");
LOG.debug("Use DataSource : {} > {}", ds.value(), point.getSignature());
DynamicDataSourceHolder.setDataSourceType(ds.value().name());
LOG.debug("[annotation.set] datasource====》{}",ds.value().name());
LOG.debug("=======================SET END=======================");
}
@After("@annotation(ds)")
public void restoreDataSource(JoinPoint point, DataSource ds) {
LOG.debug("=======================CLEAR START=======================");
LOG.debug("Revert DataSource : {} > {}", ds.value().name(), point.getSignature());
DynamicDataSourceHolder.clearDataSourceType();
LOG.debug("[annotation.remove] datasource====》{}",ds.value().name());
LOG.debug("=======================CLEAR END=======================");
}
}
7. DataSourceType.java is as follows:
package cn.edu.his.pay.dynamic.datasource;
public enum DataSourceType {
MASTER, SLAVE
}
8. DynamicDataSource.java is as follows:
package cn.edu.his.pay.dynamic.datasource;
import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;
public class DynamicDataSource extends AbstractRoutingDataSource {
@Override
protected Object determineCurrentLookupKey() {
return DynamicDataSourceHolder.getDataSourceType();
}
}
9. DynamicDataSourceHolder.java is as follows:
package cn.edu.his.pay.dynamic.datasource;
import org.springframework.util.Assert;
import cn.edu.his.pay.common.log.Logger;
public class DynamicDataSourceHolder {
private static final Logger LOG = new Logger(DynamicDataSourceHolder.class);
// 线程本地环境
private static final ThreadLocal<String> contextHolder = new ThreadLocal<String>();
// 设置数据源类型
public static void setDataSourceType(String dataSourceType) {
Assert.notNull(dataSourceType, "DataSourceType cannot be null");
contextHolder.set(dataSourceType);
LOG.debug("[this.set] datasource====》{}",dataSourceType);
}
// 获取数据源类型
public static String getDataSourceType() {
return contextHolder.get();
}
// 清除数据源类型
public static void clearDataSourceType() {
LOG.debug("[this.remove] datasource====》{}",contextHolder.get());
contextHolder.remove();
}
}
9. The basic core configuration and core code are as above, so how do we use it? For example, the point of configuring Aop in the spring-dispatcher.xml configuration is all the methods under the service package. Therefore, if you need to switch the data source to Slave, you can directly use the following annotations to configure the methods corresponding to the methods. If you do not configure annotations, you will go to Master by default.
<!-- 配置AOP，Spring是通过AOP来进行事务管理的 -->
<aop:config>
<!-- 设置pointCut表示哪些方法要加入事务处理 -->
<!-- 以下的事务是声明在DAO中，但是通常都会在Service来处理多个业务对象逻辑的关系，注入删除，更新等，此时如果在执行了一个步骤之后抛出异常
就会导致数据不完整，所以事务不应该在DAO层处理，而应该在service，这也就是Spring所提供的一个非常方便的工具，声明式事务 -->
<aop:pointcut id="allMethods" expression="(execution(* cn.edu.his.pay.service.*.*(..)))" />
<!-- 通过advisor来确定具体要加入事务控制的方法 -->
<aop:advisor advice-ref="txAdvice" pointcut-ref="allMethods" />
</aop:config>@Override
@DataSource(value = DataSourceType.SLAVE)
public int insert(Admin record) {
return adminMapper.insert(record);
}10. Question: The above configuration is based on the service as the entry point. At the same time, Baidu said that the mapper (dao layer) can be used as the entry point to do it, but I have tried several times without success. I wonder if this method can be implemented?
11. At first, I was quite confident in my implementation, but unfortunately I still did not avoid getting into the pit. When the code testers tested, they found that the function was not easy to use. After various investigations, after a day of investigation, the data connection was wrong. All kinds of data are wrong; after finding the bug, the bug was fixed, and the testers there started to test the main process payment, but it was found that it was still not easy to use, and the result was another investigation. The test library was used, and it turned out that the problem occurred in the execution of spring's nested transactions, and there was another Baidu meal without saying anything, and because there are more businesses executed in the service method, the data source switching is also more frequent, and the data The resource overhead consumed by source switching back and forth is too large, so I decided to give up and use distributed transaction management jta to implement the ACID problem of nested transactions (using jta to implement distributed transactions will be introduced in the next article), although other The method solves the problem of distributed transactions, but here I will describe the whole problem again, and I hope to discuss with you and analyze where the problem occurs?
12. In the same service method, the addition, deletion and modification of two libraries are involved, but the switching data source annotation is configured on the service method, so the data source cannot be automatically switched. The manual switching is used. The switching code is as follows:
DynamicDataSourceHolder.setDataSourceType(DataSourceType.SLAVE.name());
securityAdditionMapper.insert(securityAddition);
DynamicDataSourceHolder.clearDataSourceType();13. The nested transaction demonstration code is as follows:
@Override
@Transactional(rollbackFor = Exception.class)
public ApiCommonResultVo handlePay(){
handlePayFinish();
DynamicDataSourceHolder.setDataSourceType(DataSourceType.SLAVE.name());
securityAdditionMapper.insert(securityAddition);
DynamicDataSourceHolder.clearDataSourceType();
}
@Override
@Transactional(rollbackFor = Exception.class)
public void handlePayFinish(){
// 业务代码
}14. With the above description of the needs, I have summarized the following questions, please give the correct answers:
- Can only use spring's transaction management to achieve ACID related to multiple data source switching transactions?
- Does spring transaction support nested transactions?
- Why can't you switch data sources in spring transactions?
- For a transaction like spring, but the system completely crashes after the program runs halfway, can the ACID of the data be preserved at this time?