Configure multiple data sources in spring and spring boot

In the process of project development, sometimes we have such a demand that we need to call data in other systems, then there are multiple data sources in the system at this time, so how do we solve the problem of whether the program is running? What data source to use?

Suppose there are 2 data sources mysql and oracle in our system. There are 2 methods methodA and methodB in the system, where methodA needs to call mysql and methodB needs to use oracle, then how do we know when we call methodA or methodB Are you using mysql or oracle? My approach is to use Spring's dynamic data source routing to solve this problem.

Requirement: Some chart information should be displayed in the system, but the data of the chart is from another database, not the database used by the system itself. How to solve this problem.

Solution: Dynamically decide which data source to use in the process of running the program, and solve it with the help of Spring's abstract data source routing.

Precondition:

1. @Primary means that when there are multiple beans of the same type in the program, the ones marked with the @Primay annotation are used by default.

2. When a Connection is generated, the transaction may be opened

3. If we want to get a Connection, then we need to know which data source is used before the transaction is started. If we want to write an aspect to judge which data source to use, then we must before the transaction aspect.

4. If a variable can be obtained anywhere in the program, ThreadLocal can be used to store the variable. If it is stored in a static type variable, there will be thread safety problems.

Solutions:

1. A class AbstractRoutingDataSource is provided in Spring. There is a determineCurrentLookupKey() method in this class. In this method, a key is returned , then Spring knows which data source to use. So how does Spring know what the key you return means? Then there must be a method in this class that can associate the key with the data source, that is, the setTargetDataSources(Map<Object, Object> targetDataSources) method. Pass a map in the parameters of this method, so that the key of the map can be determinedCurrentLookupKey() The required key, the value of which is a specific data source.

2. So when do we need to set the key of the determineCurrentLookupKey() method? Then it must be the method that needs to switch the data source, that is, we need to set the key of the current data source in ThreadLocal at this time, then if we have many methods to switch the data source, does each method have to manually insert values into ThreadLocal? ? This code is too redundant, then we can customize an annotation at this time, and then write an aspect to intercept all methods with this annotation, and then set the key required by the data source in this aspect.

3. So when will our aspect be executed? We know that the transaction is opened by the Connection. If our aspect is executed after the transaction, it means nothing, that is, the data source switch fails. Therefore, the aspects we wrote ourselves need to be executed before Spring's transaction aspects. Use the @Order annotation to execute the execution order of the aspects. The smaller the value in the annotation, the first.

4. We know that our transaction needs a data source, or Jpa operation database also needs a data source, so which data source should we use as the main data source? Whether it is the data source of our system or the data source of a third-party system, this is definitely not the case. There should be dynamic decision-making during program runtime, so a class that implements AbstractRoutingDataSource is needed as the main data source.

Specific steps:

1. Introduce the pom.xml file

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.huan.springboot</groupId>
	<artifactId>springboot_20_multi_datasource</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>

	<name>springboot_20_multi_datasource</name>
	<url>http://maven.apache.org</url>

	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>1.4.3.RELEASE</version>
		<relativePath /> <!-- lookup parent from repository -->
	</parent>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
		<java.version>1.8</java.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-jpa</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
			<scope>runtime</scope>
		</dependency>
		<dependency>
			<groupId>com.oracle</groupId>
			<artifactId>ojdbc14</artifactId>
			<version>10.2.0.4.0</version>
			<scope>runtime</scope>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-configuration-processor</artifactId>
			<optional>true</optional>
		</dependency>
		<dependency>
			<groupId>org.projectlombok</groupId>
			<artifactId>lombok</artifactId>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>
</project>

Second, configure 2 data sources

1. Configure the application.yml file and add the configuration of 2 different data sources to it

spring.datasource.mysql.url=jdbc:mysql://localhost/information_schema?useUnicode=true&characterEncoding=utf-8
spring.datasource.mysql.username=root
spring.datasource.mysql.password=root
spring.datasource.mysql.driver-class-name=com.mysql.jdbc.Driver

spring.datasource.oracle.url=jdbc:oracle:thin:@127.0.0.1:1521:orcl
spring.datasource.oracle.username=system
spring.datasource.oracle.password=admin
spring.datasource.oracle.driver-class-name=oracle.jdbc.driver.OracleDriver

spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=none
spring.jpa.hibernate.naming.strategy=org.hibernate.cfg.ImprovedNamingStrategy

2. Write a MultiDataSourceConfig configuration file to configure these two data sources

package com.huan.springboot.config;

import javax.sql.DataSource;

import org.springframework.boot.autoconfigure.jdbc.DataSourceBuilder;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * Multiple data source configuration
 *
 * @describe
 * @authorhuan
 *@time March 8, 2018 - 8:47:30 pm
 */
@Configuration
public class MultiDataSourceConfig {

	@Bean
	@ConfigurationProperties("spring.datasource.mysql")
	public DataSource mysqlDataSource() {
		return DataSourceBuilder.create().build();
	}

	@Bean
	@ConfigurationProperties("spring.datasource.oracle")
	public DataSource oracleDataSource() {
		return DataSourceBuilder.create().build();
	}

}

3. Write custom annotations + data source routing

1. Customize an annotation DynamicDataSource

package com.huan.springboot.datasource;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * Dynamic data source annotations
 *
 * @describe
 * @authorhuan
 *@time March 8, 2018 - 8:55:23 pm
 */
@Target({ ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface DynamicDataSource {
	/**
	 * Default is mysql data source
	 *
	 * @return
	 */
	DynamicDataSourceTypeEnum value() default DynamicDataSourceTypeEnum.MYSQL;
}

2. Write a data source enumeration class to indicate which data sources are there to prevent manual writing from making mistakes

package com.huan.springboot.datasource;

import lombok.AllArgsConstructor;
import lombok.Getter;

/**
 * Dynamic data source type enumeration
 *
 * @describe
 * @authorhuan
 *@time March 8, 2018 - 8:55:52 pm
 */
@AllArgsConstructor
@Getter
public enum DynamicDataSourceTypeEnum {

	ORACLE("oracle", "oracle数据源"), //
	MYSQL("mysql", "currently using mysql data source");

	private String type;
	private String desc;
}

3. The program will save the above enumeration type when it needs to switch the data source, so where to save it, it must be in a thread-safe ThreadLocal, so this class DynamicDataSourceTypeHolder is required

package com.huan.springboot.datasource;

/**
 * Save the data source type of the current thread
 *
 * @describe
 * @authorhuan
 *@time March 8, 2018 - 8:59:09 pm
 */
public class DynamicDataSourceTypeHolder {

	private static final ThreadLocal<DynamicDataSourceTypeEnum> DATA_SOURCE_TYPE = new ThreadLocal<>();

	public static void setDataSourceType(DynamicDataSourceTypeEnum dataSourceTypeEnum) {
		DATA_SOURCE_TYPE.set(dataSourceTypeEnum);
	}

	public static DynamicDataSourceTypeEnum getDataSourceType() {
		return DATA_SOURCE_TYPE.get();
	}

	public static void clear() {
		DATA_SOURCE_TYPE.set(null);
	}

}

4. When we write the @DynamicDataSource annotation on the method that needs to switch the data source, we need a class to handle this annotation, so we have the following class.

package com.huan.springboot.datasource;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;

/**
 * Dynamic data source aspect
 *
 * @describe
 * @authorhuan
 *@time March 8, 2018 - 9:15:49 pm
 */
@Component
@Aspect
@Order(0)
public class DynamicDataSourceAspect {
	@Around("@annotation(dataSource)")
	public Object invoked(ProceedingJoinPoint pjp, DynamicDataSource dataSource) throws Throwable {
		DynamicDataSourceTypeHolder.setDataSourceType(dataSource.value());
		try {
			return pjp.proceed ();
		} finally {
			DynamicDataSourceTypeHolder.clear();
		}
	}
}

Notice:

1. Look at the value in @Order. This value is relatively small, which will ensure that this aspect is executed before the transaction aspect. You can delete this annotation and see the effect.

2. In this class, the key of the current data source is saved to a variable of type ThreadLocal

5. With the above content, then we definitely need to write dynamic data source routing to decide which data source key to return and which data source to use

package com.huan.springboot.datasource;

import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;

/**
 * Dynamic data source routing, which is determined by this class
 *
 * @describe
 * @authorhuan
 *@time March 8, 2018 - 9:01:58 pm
 */
public class DynamicDataSourceRouter extends AbstractRoutingDataSource {

	@Override
	protected Object determineCurrentLookupKey() {
		DynamicDataSourceTypeEnum dataSourceType = DynamicDataSourceTypeHolder.getDataSourceType();
		if (null == dataSourceType) {
			System.out.println("No data source was obtained, use the default data source.");
			return null;
		} else {
			System.out.println(dataSourceType.getDesc());
			return dataSourceType.getType();
		}
	}

Notice:

The determineCurrentLookupKey() method returns a key, so Spring knows which data source to use. So how does Spring know that? Look at the DynamicDataSourceRoteConfig configuration class below, which I haven't written yet.

6. The dynamic data source routing is written above, but the key of the data source and the specific data source have not been configured, so the configuration class below is used to configure it.

package com.huan.springboot.config;

import java.util.HashMap;
import java.util.Map;

import javax.sql.DataSource;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Lazy;
import org.springframework.context.annotation.Primary;
import org.springframework.transaction.annotation.EnableTransactionManagement;

import com.huan.springboot.datasource.DynamicDataSourceRouter;
import com.huan.springboot.datasource.DynamicDataSourceTypeEnum;

/**
 * Data source routing configuration
 *
 * @describe
 * @authorhuan
 *@time March 8, 2018 - 9:15:02 pm
 */
@Configuration
@EnableTransactionManagement
public class DynamicDataSourceRoteConfig {

	@Autowired
	@Lazy
	private DataSource mysqlDataSource;
	@Autowired
	@Lazy
	private DataSource oracleDataSource;

	@Bean
	@Primary
	public DataSource dynamicDataSourceRoute() {
		DynamicDataSourceRouter dataSourceRouter = new DynamicDataSourceRouter();
		// default is mysql data source
		dataSourceRouter.setDefaultTargetDataSource(mysqlDataSource);
		// Map a key to a specific data source
		Map<Object, Object> targetDataSources = new HashMap<>();
		targetDataSources.put(DynamicDataSourceTypeEnum.MYSQL.getType(), mysqlDataSource);
		targetDataSources.put(DynamicDataSourceTypeEnum.ORACLE.getType(), oracleDataSource);

		dataSourceRouter.setTargetDataSources(targetDataSources);
		return dataSourceRouter;
	}

}

Note: in the dynamicDataSourceRoute() method

1. If the @Primary annotation is used, this will be injected when the data source needs to be used, instead of the two data sources configured with the MultiDataSourceConfig above.

2. A default data source is set in this method, that is, that data source should be used when the key of the data source is not obtained

3. In this method, there is a dataSourceRouter.setTargetDataSources(targetDataSources); this sentence. There is an attribute targetDataSources in this , you can see that its key is associated with the specific data source.

At this point, the code for dynamically switching the data source is finished, then write a small example below to test it.

4. Examples

1. Sample code

2. View the results

Configure multiple data sources in spring and spring boot

Guess you like