SpringBoot2.X and redis Lettuce integrated stepped pit

Due

Found on a recent project issues, redis cluster clusters have a machine after the collapse, redis background service would have been an error, unable to connect to redis clusters. View redis cluster command, found redis cluster cluster is normal, spare machine has been upgraded to the slave master.
So the initial guess is spring-redis connection pooling framework in which after the collapse of a master machine, and did not refresh the connection pool in redis still hang connections is that Taiwan redis server.

By looking information and found springboot in 1.x using jedis framework, the framework and 2.x used instead Lettuce redis connection by default. Find the relevant information about Redis Cluster in Lettuce official document "Refreshing the cluster topology view"

This probably means that the inside of adaptive refresh topology (Adaptive updates) to refresh the topology (Periodic updates) is off by default, can be opened by code.

Open out

Continue to find, found Lettuce official gave open topology refresh code examples

# Example 37. Enabling periodic cluster topology view updates
RedisClusterClient clusterClient = RedisClusterClient.create(RedisURI.create("localhost", 6379));

ClusterTopologyRefreshOptions topologyRefreshOptions = ClusterTopologyRefreshOptions.builder()
                                .enablePeriodicRefresh(10, TimeUnit.MINUTES)
                                .build();

clusterClient.setOptions(ClusterClientOptions.builder()
                                .topologyRefreshOptions(topologyRefreshOptions)
                                .build());
...

clusterClient.shutdown();


# Example 38. Enabling adaptive cluster topology view updates
RedisURI node1 = RedisURI.create("node1", 6379);
RedisURI node2 = RedisURI.create("node2", 6379);

RedisClusterClient clusterClient = RedisClusterClient.create(Arrays.asList(node1, node2));

ClusterTopologyRefreshOptions topologyRefreshOptions = ClusterTopologyRefreshOptions.builder()
                                .enableAdaptiveRefreshTrigger(RefreshTrigger.MOVED_REDIRECT, RefreshTrigger.PERSISTENT_RECONNECTS)
                                .adaptiveRefreshTriggersTimeout(30, TimeUnit.SECONDS)
                                .build();

clusterClient.setOptions(ClusterClientOptions.builder()
                                .topologyRefreshOptions(topologyRefreshOptions)
                                .build());
...

clusterClient.shutdown();
复制代码

According to an example we modify our project code

import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.databind.ObjectMapper;
import io.lettuce.core.cluster.ClusterClientOptions;
import io.lettuce.core.cluster.ClusterTopologyRefreshOptions;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.data.redis.RedisProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisClusterConfiguration;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.connection.RedisNode;
import org.springframework.data.redis.connection.RedisPassword;
import org.springframework.data.redis.connection.lettuce.LettuceClientConfiguration;
import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;
import org.springframework.data.redis.connection.lettuce.LettucePoolingClientConfiguration;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;


import java.time.Duration;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

@Configuration
public class LettuceRedisConfig {


    @Autowired
    private RedisProperties redisProperties;

    /**
     * 配置RedisTemplate
     * 【Redis配置最终一步】
     *
     * @param lettuceConnectionFactoryUvPv redis连接工厂实现
     * @return 返回一个可以使用的RedisTemplate实例
     */
    @Bean
    public RedisTemplate redisTemplate(@Qualifier("lettuceConnectionFactoryUvPv") RedisConnectionFactory lettuceConnectionFactoryUvPv) {
        RedisTemplate<String, Object> template = new RedisTemplate<String, Object>();
        template.setConnectionFactory(lettuceConnectionFactoryUvPv);
        Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
        ObjectMapper om = new ObjectMapper();
        om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
        om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
        jackson2JsonRedisSerializer.setObjectMapper(om);
        StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
        // key采用String的序列化方式
        template.setKeySerializer(stringRedisSerializer);
        // hash的key也采用String的序列化方式
        template.setHashKeySerializer(stringRedisSerializer);
        // value序列化方式采用jackson
        template.setValueSerializer(jackson2JsonRedisSerializer);
        // hash的value序列化方式采用jackson
        template.setHashValueSerializer(jackson2JsonRedisSerializer);
        template.afterPropertiesSet();
        return template;
    }


    /**
     * 为RedisTemplate配置Redis连接工厂实现
     * LettuceConnectionFactory实现了RedisConnectionFactory接口
     * UVPV用Redis
     *
     * @return 返回LettuceConnectionFactory
     */
    @Bean(destroyMethod = "destroy")
    //这里要注意的是，在构建LettuceConnectionFactory 时，如果不使用内置的destroyMethod，可能会导致Redis连接早于其它Bean被销毁
    public LettuceConnectionFactory lettuceConnectionFactoryUvPv() throws Exception {

        List<String> clusterNodes = redisProperties.getCluster().getNodes();
        Set<RedisNode> nodes = new HashSet<RedisNode>();
        clusterNodes.forEach(address -> nodes.add(new RedisNode(address.split(":")[0].trim(), Integer.valueOf(address.split(":")[1]))));
        RedisClusterConfiguration clusterConfiguration = new RedisClusterConfiguration();
        clusterConfiguration.setClusterNodes(nodes);
        clusterConfiguration.setPassword(RedisPassword.of(redisProperties.getPassword()));
        clusterConfiguration.setMaxRedirects(redisProperties.getCluster().getMaxRedirects());

        GenericObjectPoolConfig poolConfig = new GenericObjectPoolConfig();
        poolConfig.setMaxIdle(redisProperties.getLettuce().getPool().getMaxIdle());
        poolConfig.setMinIdle(redisProperties.getLettuce().getPool().getMinIdle());
        poolConfig.setMaxTotal(redisProperties.getLettuce().getPool().getMaxActive());

        return new LettuceConnectionFactory(clusterConfiguration, getLettuceClientConfiguration(poolConfig));
    }

    /**
     * 配置LettuceClientConfiguration 包括线程池配置和安全项配置
     *
     * @param genericObjectPoolConfig common-pool2线程池
     * @return lettuceClientConfiguration
     */
    private LettuceClientConfiguration getLettuceClientConfiguration(GenericObjectPoolConfig genericObjectPoolConfig) {
        /*
        ClusterTopologyRefreshOptions配置用于开启自适应刷新和定时刷新。如自适应刷新不开启，Redis集群变更时将会导致连接异常！
         */
        ClusterTopologyRefreshOptions topologyRefreshOptions = ClusterTopologyRefreshOptions.builder()
                //开启自适应刷新
                //.enableAdaptiveRefreshTrigger(ClusterTopologyRefreshOptions.RefreshTrigger.MOVED_REDIRECT, ClusterTopologyRefreshOptions.RefreshTrigger.PERSISTENT_RECONNECTS)
                //开启所有自适应刷新，MOVED，ASK，PERSISTENT都会触发
                .enableAllAdaptiveRefreshTriggers()
                // 自适应刷新超时时间(默认30秒)
                .adaptiveRefreshTriggersTimeout(Duration.ofSeconds(25)) //默认关闭开启后时间为30秒
                // 开周期刷新
                .enablePeriodicRefresh(Duration.ofSeconds(20))  // 默认关闭开启后时间为60秒 ClusterTopologyRefreshOptions.DEFAULT_REFRESH_PERIOD 60  .enablePeriodicRefresh(Duration.ofSeconds(2)) = .enablePeriodicRefresh().refreshPeriod(Duration.ofSeconds(2))
                .build();
        return LettucePoolingClientConfiguration.builder()
                .poolConfig(genericObjectPoolConfig)
                .clientOptions(ClusterClientOptions.builder().topologyRefreshOptions(topologyRefreshOptions).build())
                //将appID传入连接，方便Redis监控中查看
                //.clientName(appName + "_lettuce")
                .build();
    }
}
复制代码

After the injection tool redisTemplate class to class

    @Autowired
    private RedisTemplate<String, Object> redisTemplate;
复制代码

A little closer look

We can see the parameters redisTemplate class through debug breakpoint to observe the topology refresh is turned on before is not set, we refresh cycle redisTemplate refresh and topology are false

After setting up open topology refresh, you can see our two parameter values ​​have become true

ClusterTopologyRefreshActivated parameters which are not in accordance with our increased refresh open topology is initialized to true, it is the first time the use of redisTemplate in RedisClusterClient#activateTopologyRefreshIfNeededmodification is true, are interested can look at their own debug tracing method.

Without Lettuce, use back Jedis

Of course, if you want to give up Lettuce switch jedis also possible. In Spring Boot2.X version, as long as pom.xml in, adjust dependencies references to:

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-redis</artifactId>
			<exclusions>
				<exclusion>
					<groupId>io.lettuce</groupId>
					<artifactId>lettuce-core</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		
		<dependency>
			<groupId>redis.clients</groupId>
			<artifactId>jedis</artifactId>
		</dependency>
复制代码

Lettuce replaced jedis configuration, either complete replacement of the bottom of jedis

spring:
  redis:
    jedis:
      pool:
        max-active: ${redis.config.maxTotal:1024}
        max-idle: ${redis.config.maxIdle:50}
        min-idle: ${redis.config.minIdle:1}
        max-wait: ${redis.config.maxWaitMillis:5000}
    #lettuce:
      #pool:
        #max-active: ${redis.config.maxTotal:1024}
        #max-idle: ${redis.config.maxIdle:50}
        #min-idle: ${redis.config.minIdle:1}
        #max-wait: ${redis.config.maxWaitMillis:5000}
复制代码

Jedis and Lettuce difference

Lettuce What are the advantages and disadvantages compared to the Jedis?

Lettuce Redis and Jedis are positioning the client, so of course they can be directly connected redis server.

Jedis on implementation is directly connected redis server, if not thread-safe, only this time using a connection pool in a multithreaded environment, increase physical connection for each instance of Jedis

Lettuce is based Netty connection, the connection example (StatefulRedisConnection) between multiple threads can concurrently access, should be StatefulRedisConnection is thread-safe, so a connection instance (StatefulRedisConnection) to meet the concurrent access to a multi-threaded environment, of course, also scalable design, where a connection instance can be increased enough demand connection instance.