Article directory
foreword
In actual development work, we often encounter situations where multiple data sources need to be integrated, such as connecting to multiple databases at the same time, separating reads and writes, and querying across databases. This article will introduce how to use Spring Boot to realize the integration of multiple data sources, which may be helpful for those who are new to development.
1. Basic concepts
1.1 What are multiple data sources?
Using multiple data sources in one application means that we need to switch between different data sources in order to get data from different data sources. Multiple data sources can be relational databases, NoSQL databases, flat files, XML files, etc. The advantage of using multiple data sources in an application is that we can choose the most suitable data source according to the needs of the application, thereby improving the performance and scalability of the application.
1.2 Why use multiple data sources?
The benefits of using multiple data sources in an application are many, some of which are:
- The most suitable data source can be selected according to the needs of the application, thereby improving the performance and scalability of the application.
- Risks posed by a single source of data, such as a single point of failure, can be reduced.
- It can better support data isolation and data security.
- Different business needs can be better supported.
2. How to integrate multiple data sources in Spring Boot?
2.1 Basic configuration
For the next integration example, I will introduce the actual project integration example. The project has multiple data sources of mysql and phoenix. Then describe it in detail. Because MybatisPlus is integrated in the project, there will also be some related configurations. First , we need to add the corresponding dependencies in the pom.xml file. The following are commonly used data source dependencies:
<!-- 数据源 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<!-- MySQL 数据库 -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<!-- phoenix 数据库 -->
<dependency>
<groupId>org.apache.phoenix</groupId>
<artifactId>phoenix-core</artifactId>
</dependency>
<!-- mybatis-plus -->
<dependency>
<groupId>com.baomidou</groupId>
<artifactId>mybatis-plus-boot-starter</artifactId>
</dependency>
Next, we need to configure the connection information of each data source in the application.properties or application.yml file. Here is an example configuration:
#数据库配置
datasource:
#mysql数据源配置
mysql:
type: com.zaxxer.hikari.HikariDataSource
jdbc-url: jdbc:p6spy:mysql://xxxx:3306/xxxx?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=GMT%2B8&allowPublicKeyRetrieval=true
username: xxxx
password: xxxxx
# Hikari 连接池配置
# 最小空闲连接数量
hikari:
minimum-idle: 5
# 空闲连接存活最大时间，默认600000（10分钟）
idle-timeout: 180000
# 连接池最大连接数，默认是10
maximum-pool-size: 10
# 此属性控制从池返回的连接的默认自动提交行为,默认值：true
auto-commit: true
# 连接池名称
pool-name: xxxx-HikariCP
# 此属性控制池中连接的最长生命周期，值0表示无限生命周期，默认1800000即30分钟
max-lifetime: 1800000
# 数据库连接超时时间,默认30秒，即30000
connection-timeout: 30000
connection-test-query: SELECT 1
#phoenix数据源配置
phoenix:
driver: org.apache.phoenix.jdbc.PhoenixDriver
jdbcUrl: jdbc:phoenix:xxxxx:xxxx
user:
password:
connectionProperties:
isNamespaceMappingEnabled: true
mapSystemTablesToNamespace: true
schema: 'xxxxxx'
maximumPoolSize: 128
maxLifetime: 1200000
After that, we need to create configuration classes for multiple data sources, as well as some basic configurations. Here is sample code:
@EnableTransactionManagement
@Configuration
@MapperScan("com.xxxx.mapper")
public class DataSourceConfig {
@Autowired
private PhoenixProperties phoenixProperties;
@Bean
public PaginationInterceptor paginationInterceptor() {
return new PaginationInterceptor().setDialectType("mysql");
}
@Bean(name = "mysqlDateSource")
@ConfigurationProperties(prefix = "spring.datasource.mysql")
public HikariDataSource mysqlDateSource() {
return DataSourceBuilder.create().type(HikariDataSource.class).build();
}
@Bean(name = "phoenixDataSource")
public HikariDataSource phoenixDataSource() {
HikariDataSource dataSource = new HikariDataSource();
dataSource.setDriverClassName(phoenixProperties.getDriver());
dataSource.setJdbcUrl(phoenixProperties.getJdbcUrl());
dataSource.setUsername(phoenixProperties.getUser());
dataSource.setPassword(phoenixProperties.getPassword());
Properties properties = new Properties();
properties.setProperty("phoenix.schema.isNamespaceMappingEnabled", phoenixProperties.getIsNamespaceMappingEnabled());
properties.setProperty("phoenix.schema.mapSystemTablesToNamespace", phoenixProperties.getMapSystemTablesToNamespace());
properties.setProperty("schema", phoenixProperties.getSchema());
properties.setProperty("maximum-pool-size", phoenixProperties.getMaximumPoolSize());
properties.setProperty("max-lifetime", phoenixProperties.getMaxLifetime());
dataSource.setDataSourceProperties(properties);
return dataSource;
}
/**
* 动态数据源配置
*
* @return DataSource
*/
@Bean
@Primary
public DataSource multipleDataSource(@Qualifier("mysqlDateSource") DataSource mysqlDateSource, @Qualifier("phoenixDataSource") DataSource phoenixDataSource) {
MultipleDataSource multipleDataSource = new MultipleDataSource();
Map<Object, Object> targetDataSources = Maps.newHashMap();
targetDataSources.put(DataSourceEnum.MYSQL.getValue(), mysqlDateSource);
targetDataSources.put(DataSourceEnum.PHOENIX.getValue(), phoenixDataSource);
//添加数据源
multipleDataSource.setTargetDataSources(targetDataSources);
//设置默认数据源
multipleDataSource.setDefaultTargetDataSource(mysqlDateSource);
return multipleDataSource;
}
@Bean(name = "sqlSessionFactory")
public SqlSessionFactory sqlSessionFactory() throws Exception {
MybatisSqlSessionFactoryBean sqlSessionFactory = new MybatisSqlSessionFactoryBean();
sqlSessionFactory.setDataSource(multipleDataSource(mysqlDateSource(), phoenixDataSource()));
ResourceLoader loader = new DefaultResourceLoader();
String resource = "classpath:mybatis-config.xml";
sqlSessionFactory.setConfigLocation(loader.getResource(resource));
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
sqlSessionFactory.setMapperLocations(resolver.getResources("classpath*:/mapper/*.xml"));
sqlSessionFactory.setTypeAliasesPackage("com.huitongjy.oms.web.entity");
sqlSessionFactory.setTypeEnumsPackage("com.huitongjy.oms.common.enums");
//关键代码 设置 MyBatis-Plus 分页插件
Interceptor[] plugins = {
paginationInterceptor()};
sqlSessionFactory.setPlugins(plugins);
return sqlSessionFactory.getObject();
}
}
MultipleDataSource class
public class MultipleDataSource extends AbstractRoutingDataSource {
@Override
protected Object determineCurrentLookupKey() {
return DataSourceContextHolder.getDataSource();
}
}
class DataSourceContextHolder {
private static final ThreadLocal<String> CONTEXT_HOLDER = new InheritableThreadLocal<>();
/**
* 设置数据源
*
* @param db String
*/
static void setDataSource(String db) {
CONTEXT_HOLDER.set(db);
}
/**
* 取得当前数据源
*
* @return String
*/
static String getDataSource() {
return CONTEXT_HOLDER.get();
}
/**
* 清除上下文数据
*/
static void clear() {
CONTEXT_HOLDER.remove();
}
}
The configuration of mybatis-config.xml is as follows:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration
PUBLIC "-//mybatis.org//DTD Config 3.0//EN"
"http://mybatis.org/dtd/mybatis-3-config.dtd">
<configuration>
<settings>
<setting name="cacheEnabled" value="false"/>
<setting name="lazyLoadingEnabled" value="true"/>
<setting name="multipleResultSetsEnabled" value="true"/>
<setting name="useColumnLabel" value="true"/>
<setting name="useGeneratedKeys" value="false"/>
<setting name="autoMappingBehavior" value="PARTIAL"/>
<setting name="autoMappingUnknownColumnBehavior" value="WARNING"/>
<setting name="defaultExecutorType" value="SIMPLE"/>
<setting name="defaultStatementTimeout" value="25"/>
<setting name="defaultFetchSize" value="100"/>
<setting name="safeRowBoundsEnabled" value="false"/>
<setting name="mapUnderscoreToCamelCase" value="true"/>
<setting name="localCacheScope" value="SESSION"/>
<setting name="jdbcTypeForNull" value="OTHER"/>
<setting name="lazyLoadTriggerMethods" value="equals,clone,hashCode,toString"/>
</settings>
</configuration>
In the above code, we use @ConfigurationProperties(prefix = "spring.datasource.mysql") to bind the properties in the configuration file to the HikariDataSource object of mysql. Similarly, we also bind for phoenixDataSource.
These two data sources are connected to different databases respectively. The @Primary annotation represents the primary data source and is used to decide which data source to use at runtime. We created a method called multipleDataSource, which adds a data source and sets the default data source.
Finally, load multiple data sources and some configuration files in SqlSessionFactory so that the database connection can be managed and configured.
2.2 Item Code
After configuration, we only need to configure the data source on the phoenix interface, the code is as follows:
public interface TestDetailMapper {
/**
* 保存
* @param testDetail
*/
@DataSource(DataSourceEnum.PHOENIX)
@Insert(" upsert into xxx_xxx (xxxxx) " +
" values (#{xxxx})")
void insert(@Param("xxx") TestDetail testDetail);
/**
* 获取内容
*
* @param xxxx
* @return xxx
*/
@Results(value = {
@Result(property = "xxxx", column = "xxxxx")
})
@DataSource(DataSourceEnum.PHOENIX)
@Select("SELECT XXXX FROM XXX WHERE XXX=#{XXXX}")
TestDetail findByKey(@Param("XXXX") String XXXX);
}
Through the above code, we have successfully realized the integration of multiple data sources.
2.3 Precautions
(1) The primary data source must be specified, that is, add a @Primary annotation to the bean injection of the primary data source, indicating that multiple objects of the same class are injected, and the object with the annotation @Primary is preferred.
(2) It is best to separate the dao and mapper files of different data sources into different package paths and file paths, so that the configuration file mapping can be configured separately, otherwise errors will occur.