Recently task light, free learning learning technology, then to study the separate read and write if realized. Here with the process of recording the next blog, on the one hand for future viewing, but also to share to you (the information online is really mostly copy to copy ,, is not formatted, looking really uncomfortable).
Complete code: https://github.com/FleyX/demo-project/tree/master/dxfl
1. Background
A project database the most basic but also the most mainstream stand-alone database, reading and writing in a library. When the user gradually increased, stand-alone database can not meet the performance requirements, they will read and write separation transformation (suitable for reading and writing less), writes a library, read multiple libraries usually do a database cluster, open the main from backup, a master multi-slave, to improve the reading performance. When the user more separate read and write can not be met, we need a distributed database (that may later be learning how to get).
The separate read and write to achieve under normal circumstances, a database cluster first thing to do from a multi-master, but also the need for data synchronization. This is a record of how many times to establish the configuration of a master with a mysql, how the code recorded at a write level to achieve separation.
2, set up a multi-master database from a cluster
From the master virtual machine backup requires multiple virtual machines, I was wmware complete clone multiple instances, note directly lead to the same clone uuid each database, need to be modified for different uuid. Reference to modify this: click to jump .
Main Library Configuration
Master database (master) for reading in a new user logs from the primary database binary database (slave), sql statement is as follows:
mysql> CREATE USER 'repl'@'%' IDENTIFIED BY '123456';#创建用户 mysql> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'%';#分配权限 mysql>flush privileges; #刷新权限At the same time modify the configuration file mysql open the binary log, add the following:
[mysqld] server-id=1 log-bin=master-bin log-bin-index=master-bin.indexThen restart the database, using the
show master status;statement view the main state library, as follows:
From the Library Configuration
Also to add a few lines of configuration:
[mysqld] server-id=2 relay-log-index=slave-relay-bin.index relay-log=slave-relay-binThen restart the database, the primary database is connected using a statement:
CHANGE MASTER TO MASTER_HOST='192.168.226.5', MASTER_USER='root', MASTER_PASSWORD='123456', MASTER_LOG_FILE='master-bin.000003', MASTER_LOG_POS=154;Then run
start slave;open backed, normally as shown below: Slave_IO_Running Slave_SQL_Running and are to yes.
This step can open multiple from the library.
By default, all backup operations is the main library will be backed up to the library, may actually need to ignore certain libraries, you can add the following configuration in the master library:
# 不同步哪些数据库
binlog-ignore-db = mysql
binlog-ignore-db = test
binlog-ignore-db = information_schema
# 只同步哪些数据库，除此之外，其他不同步
binlog-do-db = game3, the code level for separate read and write
Code environment springboot + mybatis + druib connection pool. Want to separate read and write a plurality of data sources needs to be configured, during a write operation is to select a write data source, data source selecting read read operation. There are two key points:
- How to switch data sources
- How to choose the correct data sources according to different methods
1), how to switch the data source
It is usually used when springboot its default configuration, only need to define the connection properties are on the line in the configuration file, but now we need to configure their own, spring to support multiple data sources, multiple datasource in a HashMap TargetDataSourcein by dertermineCurrentLookupKeyacquiring key data source to sleep which set to use. So our goal is very clear, to create multiple datasource into TargetDataSource while rewriting dertermineCurrentLookupKey methods to determine which key.
2) How to select a data source
Notes transactions are generally in the Service layer, so that at the start of this service method is called to determine the source of the data, what is the general method of operation can be done before the start of the implementation of a way to do that? I believe you have thought of that section . How cut two ways:
- Type annotation, annotation defines a read process of the data is marked using a read library
- Method name, write the name of the method according to the cut-off point, such as getXXX with Reading Library, setXXX with write library
3), coding
a, write configuration files, configuration information of the two data sources
Only the required information, others have default settings
mysql:
datasource:
#读库数目
num: 1
type-aliases-package: com.example.dxfl.dao
mapper-locations: classpath:/mapper/*.xml
config-location: classpath:/mybatis-config.xml
write:
url: jdbc:mysql://192.168.226.5:3306/test?useUnicode=true&characterEncoding=utf-8&useSSL=true
username: root
password: 123456
driver-class-name: com.mysql.jdbc.Driver
read:
url: jdbc:mysql://192.168.226.6:3306/test?useUnicode=true&characterEncoding=utf-8&useSSL=true
username: root
password: 123456
driver-class-name: com.mysql.jdbc.Driverb, write DbContextHolder class
This class is used to set the database category, which has a ThreadLocal to hold each thread is to use the library to read or write library. code show as below:
/**
* Description 这里切换读/写模式
* 原理是利用ThreadLocal保存当前线程是否处于读模式（通过开始READ_ONLY注解在开始操作前设置模式为读模式，
* 操作结束后清除该数据，避免内存泄漏，同时也为了后续在该线程进行写操作时任然为读模式
* @author fxb
* @date 2018-08-31
*/
public class DbContextHolder {
private static Logger log = LoggerFactory.getLogger(DbContextHolder.class);
public static final String WRITE = "write";
public static final String READ = "read";
private static ThreadLocal<String> contextHolder= new ThreadLocal<>();
public static void setDbType(String dbType) {
if (dbType == null) {
log.error("dbType为空");
throw new NullPointerException();
}
log.info("设置dbType为：{}",dbType);
contextHolder.set(dbType);
}
public static String getDbType() {
return contextHolder.get() == null ? WRITE : contextHolder.get();
}
public static void clearDbType() {
contextHolder.remove();
}
}c, the method override determineCurrentLookupKey
spring at the beginning of database operations will be determined by this method which database, so we call up here DbContextHolder class getDbType()method to get the current operation category, while load balancing reading library code is as follows:
public class MyAbstractRoutingDataSource extends AbstractRoutingDataSource {
@Value("${mysql.datasource.num}")
private int num;
private final Logger log = LoggerFactory.getLogger(this.getClass());
@Override
protected Object determineCurrentLookupKey() {
String typeKey = DbContextHolder.getDbType();
if (typeKey == DbContextHolder.WRITE) {
log.info("使用了写库");
return typeKey;
}
//使用随机数决定使用哪个读库
int sum = NumberUtil.getRandom(1, num);
log.info("使用了读库{}", sum);
return DbContextHolder.READ + sum;
}
}d, write configuration class
Due to read and write separation, can no longer be the default configuration springboot, we need to be configured manually. Generating a first data source, the data source generated automatically using @ConfigurProperties:
/**
* 写数据源
*
* @Primary 标志这个 Bean 如果在多个同类 Bean 候选时，该 Bean 优先被考虑。
* 多数据源配置的时候注意，必须要有一个主数据源，用 @Primary 标志该 Bean
*/
@Primary
@Bean
@ConfigurationProperties(prefix = "mysql.datasource.write")
public DataSource writeDataSource() {
return new DruidDataSource();
}Read similar data source, pay attention to how many how many will read library read data source settings, Bean called read + serial number.
Then set the data source, using our prior written MyAbstractRoutingDataSource class
/**
* 设置数据源路由，通过该类中的determineCurrentLookupKey决定使用哪个数据源
*/
@Bean
public AbstractRoutingDataSource routingDataSource() {
MyAbstractRoutingDataSource proxy = new MyAbstractRoutingDataSource();
Map<Object, Object> targetDataSources = new HashMap<>(2);
targetDataSources.put(DbContextHolder.WRITE, writeDataSource());
targetDataSources.put(DbContextHolder.READ+"1", read1());
proxy.setDefaultTargetDataSource(writeDataSource());
proxy.setTargetDataSources(targetDataSources);
return proxy;
}Then you need to set sqlSessionFactory
/**
* 多数据源需要自己设置sqlSessionFactory
*/
@Bean
public SqlSessionFactory sqlSessionFactory() throws Exception {
SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
bean.setDataSource(routingDataSource());
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
// 实体类对应的位置
bean.setTypeAliasesPackage(typeAliasesPackage);
// mybatis的XML的配置
bean.setMapperLocations(resolver.getResources(mapperLocation));
bean.setConfigLocation(resolver.getResource(configLocation));
return bean.getObject();
}And finally had to configure the transactional Otherwise, the transaction does not take effect
/**
* 设置事务，事务需要知道当前使用的是哪个数据源才能进行事务处理
*/
@Bean
public DataSourceTransactionManager dataSourceTransactionManager() {
return new DataSourceTransactionManager(routingDataSource());
}4), select a data source
Multiple data sources configured, but the code level, how to choose Select Data Source it? We describe two approaches here:
a, annotation style
First, the definition of a read-only annotation, this annotation method is used to read the library, other libraries use to write, if the project is to transform the way the reader can be separated using this method, without modifying the business code, just add on a read-only service method comment It can be.
@Target({ElementType.METHOD,ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
public @interface ReadOnly {
} Then write a section which switches the data source used, to ensure that this section rewriting getOrder higher priority than the priority of the transaction section, in classes with the start @EnableTransactionManagement(order = 10), to the following code:
@Aspect
@Component
public class ReadOnlyInterceptor implements Ordered {
private static final Logger log= LoggerFactory.getLogger(ReadOnlyInterceptor.class);
@Around("@annotation(readOnly)")
public Object setRead(ProceedingJoinPoint joinPoint,ReadOnly readOnly) throws Throwable{
try{
DbContextHolder.setDbType(DbContextHolder.READ);
return joinPoint.proceed();
}finally {
//清楚DbType一方面为了避免内存泄漏，更重要的是避免对后续在本线程上执行的操作产生影响
DbContextHolder.clearDbType();
log.info("清除threadLocal");
}
}
@Override
public int getOrder() {
return 0;
}
}b, method name of formula
This method is not allowed to comment, but need a method according to certain rules written in the name of service, and then set up the database by category section, for example, setXXXto write, getXXXto read, I do not write the code, should know how to write.
4, the test
The code is written to try what the outcome, the following is run shot:
Just separate read and write database extensions of a temporary solution, not once and for all, as the load further increases, only one library for writing is definitely not enough, and single-table database is capped, mysql up to thousands of levels the data can be kept better query performance. It will eventually become - sub-library sub-table architecture. Sub-library sub-table can look at this one: https://www.tapme.top/blog/detail/2019-03-20-10-38
Original article published on: www.tapme.top/blog/detail/2018-09-10-10-38