Redis Cluster cluster with integrated spring boot 2

Set up Redis Cluster Cluster

As used herein, redis-5.0.5, redis installed in / soft / redis directory, to be the new / soft / redis / data directory.

Install ruby

wget https://cache.ruby-lang.org/pub/ruby/2.6/ruby-2.6.3.tar.gz
tar -zxvf ruby-2.6.3.tar.gz
cd ruby-2.6.3/ 
./configure -prefix=/usr/local/ruby
make
make install
cd /usr/local/ruby/
cp bin/ruby /usr/local/bin/
cp bin/gem /usr/local/bin/
ruby --version

Creating redis Configuration

cd /soft/redis

I came config / pens-7000.conf

port 7000
protected-mode no
daemonize yes
dir "/soft/redis/data"
dbfilename "dump-7000.rdb"
logfile "log-7000.log"

cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-require-full-coverage no

Detailed configuration
cluster-enabled yes start cluster cluster
cluster-require-full-coverage no one node is unavailable, the cluster still running

Creating another 5 parts Configuration

sed 's/7000/7001/g' config/redis-7000.conf > config/redis-7001.conf
sed 's/7000/7002/g' config/redis-7000.conf > config/redis-7002.conf
sed 's/7000/7003/g' config/redis-7000.conf > config/redis-7003.conf
sed 's/7000/7004/g' config/redis-7000.conf > config/redis-7004.conf
sed 's/7000/7005/g' config/redis-7000.conf > config/redis-7005.conf

It started six instances redis

redis-server config/redis-7000.conf
redis-server config/redis-7001.conf
redis-server config/redis-7002.conf
redis-server config/redis-7003.conf
redis-server config/redis-7004.conf
redis-server config/redis-7005.conf

Creating a cluster, redis-v5 version no longer uses redis-trib.rb to create a cluster, instead of using redis-cli --cluster (192.168.4.176 is ip linux server)

redis-cli --cluster create  --cluster-replicas 1 \
192.168.4.176:7000 192.168.4.176:7001 192.168.4.176:7002 192.168.4.176:7003 192.168.4.176:7004 192.168.4.176:7005

Creation process requires you to enter a confirmation yes
yes

Open a client

redis-cli -p 7000

View cluster status
cluster info

View from the primary cluster node, slot allocation
cluster nodes

In fact the new cluster cluster is not the focus of this article, there are many online tutorials this, cluster clusters and spring boot 2 Integration is the focus of this article.

spring boot 2 integrated redis cluster

pom.xml configuration

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.1.6.RELEASE</version>
    <relativePath/> <!-- lookup parent from repository -->
</parent>

其他配置省略不写了
<dependencies>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <!-- cluster集群模式必须使用jedis 2.X，不然无法做到故障迁移 -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-redis</artifactId>
        <exclusions>
            <exclusion>
                <groupId>io.lettuce</groupId>
                <artifactId>lettuce-core</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>redis.clients</groupId>
        <artifactId>jedis</artifactId>
        <version>2.10.2</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
    </dependency>
</dependencies>

default spring boot 2 as lettuce-core redis the client. I started using Redis lettuce-core connector, when I kill off a master node (node ​​7000), Redis from the server to complete the master switch failover. spring boot 2 but still connect 7000 nodes, resulting in fall 7000 node key slot requests have been an error. It is estimated that lettuce-core client does not re-establish the connection pool cache, so the client can not do failover. Then switch to jedis 2.X version, spring boot 2 project was done failover, do not connect 7000 nodes. Note jedis 3.X versions cause spring-boot 2 can not be started.

application.properties Configuration

spring.redis.timeout=2000
spring.redis.jedis.pool.max-active=8
spring.redis.jedis.pool.max-idle=8
spring.redis.jedis.pool.min-idle=0
spring.redis.jedis.pool.max-wait=1000

spring.redis.cluster.nodes=192.168.4.176:7000, 192.168.4.176:7001, 192.168.4.176:7002, 192.168.4.176:7003, 192.168.4.176:7004, 192.168.4.176:7005
# 重定向次数
spring.redis.cluster.max-redirects=5

Where there is a very pit father, written after configuration application.properties, is only able to connect to the cluster, the client can not do failover, also need to manually write a JedisConnectionFactory configuration class, then JedisConnectionFactory set to factory configuration RedisTemplate The following is the code.

@Configuration
public class RedisConfigJedis {

    @Value("${spring.redis.timeout}")
    private Integer redisTimeout;
    @Value("${spring.redis.jedis.pool.max-active}")
    private Integer poolMaxActive;
    @Value("${spring.redis.jedis.pool.max-idle}")
    private Integer poolMaxIdle;
    @Value("${spring.redis.jedis.pool.min-idle}")
    private Integer poolMinIdle;
    @Value("${spring.redis.jedis.pool.max-wait}")
    private Integer poolMaxWait;
    @Value("${spring.redis.cluster.nodes}")
    private List<String> clusterNodes;
    @Value("${spring.redis.cluster.max-redirects}")
    private Integer clusterMaxRedirects;

    @Bean
    public RedisConnectionFactory redisConnectionFactory() {
        JedisPoolConfig poolConfig = new JedisPoolConfig();
        poolConfig.setMaxTotal(poolMaxActive);
        poolConfig.setMaxIdle(poolMaxIdle);
        poolConfig.setMinIdle(poolMinIdle);
        poolConfig.setMaxWaitMillis(poolMaxWait);
        JedisClientConfiguration clientConfig = JedisClientConfiguration.builder()
                .usePooling().poolConfig(poolConfig).and().readTimeout(Duration.ofMillis(redisTimeout)).build();

        // cluster模式
        RedisClusterConfiguration redisConfig = new RedisClusterConfiguration();
        redisConfig.setMaxRedirects(clusterMaxRedirects);
        for (String ipPort :clusterNodes){
            String[] ipPortArr = ipPort.split(":");
            redisConfig.clusterNode(ipPortArr[0], Integer.parseInt(ipPortArr[1]));
        }

        return new JedisConnectionFactory(redisConfig, clientConfig);
    }

    @Bean
    public RedisTemplate redisTemplate(RedisConnectionFactory redisConnectionFactory) {
        RedisTemplate template = new RedisTemplate();
        template.setConnectionFactory(redisConnectionFactory);

        //key使用StringRedisSerializer
        StringRedisSerializer strSerializer = new StringRedisSerializer();
        template.setKeySerializer(strSerializer);
        template.setHashKeySerializer(strSerializer);

        Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
        ObjectMapper om = new ObjectMapper();
        om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
        om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
        jackson2JsonRedisSerializer.setObjectMapper(om);

        //value使用Jackson2JsonRedisSerializer
        template.setValueSerializer(jackson2JsonRedisSerializer);
        template.setHashValueSerializer(jackson2JsonRedisSerializer);

        return template;
    }
}

Under summary, when redis kill off a cluster master node (node ​​7000), the main node from a node promoted to 7003, the node 7000 offline. redis cluster failover no problem, but spring boot project at this time to clear out old redis connection pool cache, cache re-establish the connection pool, so as to put 7,000 nodes already invalid connection pool removed, no longer connected to the node 7000. spring boot 2 versions do failover, pay attention to the following points.

1, pom.xml jedis 2.X version dependencies introduced. jedis 3.X version does not work.

2, the configuration application.properties JedisConnectionFactory again to the configuration, RedisTemplate JedisConnectionFactory used as the connection factory.

Of course, it may be where I made a mistake, we welcome the discussion message.

Next, write a class test cycle call redis

@RunWith(SpringRunner.class)
@SpringBootTest
public class Test01 {

    @Autowired
    RedisTemplate redisTemplate;

    @Test
    public void test001(){
        while (true){
            try {
                String key = "test:" + new Date().getTime();
                redisTemplate.opsForValue().set(key, new Date().getTime());
                TimeUnit.MILLISECONDS.sleep(100L);
                System.out.println(redisTemplate.opsForValue().get(key));
            }catch (Exception e){
                e.printStackTrace();
            }
        }
    }
}

Run the test class

Open a redis-cli

redis-cli -p 7002

View cluster node configuration

cluster nodes

7001 is the master node, node 7005 is from 7001

Process kill the master node 7001

ps -ef | grep Redis server

kill -9 master process ID

At this spring boot project will be reported abnormal, after a short time, the project is operating normally.

View redis cluster node again

7005 was promoted to master, 7001 off the assembly line.

7001 node starts again, 7001 will become 7005 from the node.

When 7001,7005 pair of master and slave nodes are offline due to be responsible for 5491-10933 slot end nodes are down, get, the request will be set to this side of the slot error.