Instead of using the scan instruction keys in RedisTemplate

keys *Do not mess with this command in a production environment. Especially the huge data situation. Because Keys will lead Redis lock, and increase CPU usage on the Redis. Many companies are prohibited the operation and maintenance of this command

When you need to scan key, matching the key they need, you can use the scancommand

scanHelper operations to achieve

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import java.util.function.Consumer;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.connection.RedisConnection;
import org.springframework.data.redis.core.Cursor;
import org.springframework.data.redis.core.ScanOptions;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Component;

@Component
public class RedisHelper {
    
    @Autowired
    private StringRedisTemplate stringRedisTemplate;
    
    /**
     * scan 实现
     * @param pattern   表达式
     * @param consumer  对迭代到的key进行操作
     */
    public void scan(String pattern, Consumer<byte[]> consumer) {
        this.stringRedisTemplate.execute((RedisConnection connection) -> {
            try (Cursor<byte[]> cursor = connection.scan(ScanOptions.scanOptions().count(Long.MAX_VALUE).match(pattern).build())) {
                cursor.forEachRemaining(consumer);
                return null;
            } catch (IOException e) {
                e.printStackTrace();
                throw new RuntimeException(e);
            }
        });
    }

    /**
     * 获取符合条件的key
     * @param pattern   表达式
     * @return
     */
    public List<String> keys(String pattern) {
        List<String> keys = new ArrayList<>();
        this.scan(pattern, item -> {
            //符合条件的key
            String key = new String(item,StandardCharsets.UTF_8);
            keys.add(key);
        });
        return keys;
    }
}

But there is a problem: can not move cursor, can only scan once, and prone to error redis link

First understand the next scan, hscan, sscan, zscan

http://doc.redisfans.com/key/scan.html

Why keys unsafe?

• keys of the operation will cause a temporary database is locked, other requests will be blocked; large volume of business will go wrong when

Spring RedisTemplate实现scan

1. hscan sscan zscan

• Examples of the "field" is the value of the redis key, for the "field" in the hash key from the lookup i.e.
• redisTemplate the opsForHash, opsForSet, opsForZSet may correspond sscan, hscan, zscan
• Examples of this online course is actually not right, because there is no traversal holding a cursor, just scan a check
• Use can be lazy .count(Integer.MAX_VALUE), check back all at once; but this child and keys so what difference does it make? Funny Face & doubt face
• You may be used (JedisCommands) connection.getNativeConnection()hscan of, sscan, zscan cursor traversal implemented method, with reference to section 2.2 below

try {
    Cursor<Map.Entry<Object,Object>> cursor = redisTemplate.opsForHash().scan("field",
    ScanOptions.scanOptions().match("*").count(1000).build());
    while (cursor.hasNext()) {
        Object key = cursor.next().getKey();
        Object valueSet = cursor.next().getValue();
    }
    //关闭cursor
    cursor.close();
} catch (IOException e) {
    e.printStackTrace();
}

• cursor.close (); cursor must be closed, otherwise the connection will grow; you can use the client lists``info clients``info statscommand to view the client connection status, you will find there has been a scan operation
• redisTemplate.execute we usually use will take the initiative to release the connection, you can view the source code confirm

client list
......
id=1531156 addr=xxx:55845 fd=8 name= age=80 idle=11 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=scan
......
org.springframework.data.redis.core.RedisTemplate#execute(org.springframework.data.redis.core.RedisCallback<T>, boolean, boolean)

finally {
    RedisConnectionUtils.releaseConnection(conn, factory);
}

2. scan

2.1 Internet mostly to this example

• This connection.scan could not move cursor, you can only scan once

public Set<String> scan(String matchKey) {
    Set<String> keys = redisTemplate.execute((RedisCallback<Set<String>>) connection -> {
        Set<String> keysTmp = new HashSet<>();
        Cursor<byte[]> cursor = connection.scan(new ScanOptions.ScanOptionsBuilder().match("*" + matchKey + "*").count(1000).build());
        while (cursor.hasNext()) {
            keysTmp.add(new String(cursor.next()));
        }
        return keysTmp;
    });

    return keys;
}

2.2 MultiKeyCommands

• Get connection.getNativeConnection; connection.getNativeConnection()actual object is Jedis (debug can be seen), Jedis implements many interfaces

public class Jedis extends BinaryJedis implements JedisCommands, MultiKeyCommands, AdvancedJedisCommands, ScriptingCommands, BasicCommands, ClusterCommands, SentinelCommands 

• When scan.getStringCursor () exists and is not 0, the acquisition has been to move the cursor

public Set<String> scan(String key) {
    return redisTemplate.execute((RedisCallback<Set<String>>) connection -> {
        Set<String> keys = Sets.newHashSet();

        JedisCommands commands = (JedisCommands) connection.getNativeConnection();
        MultiKeyCommands multiKeyCommands = (MultiKeyCommands) commands;

        ScanParams scanParams = new ScanParams();
        scanParams.match("*" + key + "*");
        scanParams.count(1000);
        ScanResult<String> scan = multiKeyCommands.scan("0", scanParams);
        while (null != scan.getStringCursor()) {
            keys.addAll(scan.getResult());
            if (!StringUtils.equals("0", scan.getStringCursor())) {
                scan = multiKeyCommands.scan(scan.getStringCursor(), scanParams);
                continue;
            } else {
                break;
            }
        }

        return keys;
    });
}

Divergent thinking

cursor does not close, in the end who is blocked, it is Redis

• During the test, I basically just launched a dozen scan operations, did not close the cursor, subsequent requests are stuck

redis side analysis

• client lists``info clients``info statsView
  found only a dozen the number of connections, there is no blocking and rejected the connection
• config get maxclientsThe maximum number of connections allowed queries redis 10000

1) "maxclients"
2) "10000"`

• redis-cliOn the other machines can also log in directly operate

In summary, redis itself is not stuck

Application SIDE

• netstatCheck the connection and the redis, 6333 is redis port; there has been a connection

➜  ~ netstat -an | grep 6333
netstat -an | grep 6333
tcp4       0      0  xx.xx.xx.aa.52981      xx.xx.xx.bb.6333     ESTABLISHED
tcp4       0      0  xx.xx.xx.aa.52979      xx.xx.xx.bb.6333     ESTABLISHED
tcp4       0      0  xx.xx.xx.aa.52976      xx.xx.xx.bb.6333     ESTABLISHED
tcp4       0      0  xx.xx.xx.aa.52971      xx.xx.xx.bb.6333     ESTABLISHED
tcp4       0      0  xx.xx.xx.aa.52969      xx.xx.xx.bb.6333     ESTABLISHED
tcp4       0      0  xx.xx.xx.aa.52967      xx.xx.xx.bb.6333     ESTABLISHED
tcp4       0      0  xx.xx.xx.aa.52964      xx.xx.xx.bb.6333     ESTABLISHED
tcp4       0      0  xx.xx.xx.aa.52961      xx.xx.xx.bb.6333     ESTABLISHED

• jstackView application stack information
  found WAITING many threads, all acquired in connection redis
  so basically it can be concluded that the application of redis thread pool is full

"http-nio-7007-exec-2" #139 daemon prio=5 os_prio=31 tid=0x00007fda36c1c000 nid=0xdd03 waiting on condition [0x00007000171ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c26ef560> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:590)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:441)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:362)
        at redis.clients.util.Pool.getResource(Pool.java:49)
        at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226)
        at redis.clients.jedis.JedisPool.getResource(JedisPool.java:16)
        at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:276)
        at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:469)
        at org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:132)
        at org.springframework.data.redis.core.RedisTemplate.executeWithStickyConnection(RedisTemplate.java:371)
        at org.springframework.data.redis.core.DefaultHashOperations.scan(DefaultHashOperations.java:244)

In summary, the application side is stuck

Follow-up

• After a noon, redis client listsdisplay scan connection is still no release; the application thread is still stuck in the state
• Inspection config get timeout, redis timeout is not provided, may be config set timeout xxxprovided, in seconds; the redis however a timeout, redis connection is released, the same application or stuck

1) "timeout"
2) "0"

• netstatView and redis connection, 6333 is redis port; connection from ESTABLISHED become CLOSE_WAIT;
• jstackAnd performance as the original card inJedisConnectionFactory.getConnection

➜  ~ netstat -an | grep 6333
netstat -an | grep 6333
tcp4       0      0  xx.xx.xx.aa.52981      xx.xx.xx.bb.6333     CLOSE_WAIT
tcp4       0      0  xx.xx.xx.aa.52979      xx.xx.xx.bb.6333     CLOSE_WAIT
tcp4       0      0  xx.xx.xx.aa.52976      xx.xx.xx.bb.6333     CLOSE_WAIT
tcp4       0      0  xx.xx.xx.aa.52971      xx.xx.xx.bb.6333     CLOSE_WAIT
tcp4       0      0  xx.xx.xx.aa.52969      xx.xx.xx.bb.6333     CLOSE_WAIT
tcp4       0      0  xx.xx.xx.aa.52967      xx.xx.xx.bb.6333     CLOSE_WAIT
tcp4       0      0  xx.xx.xx.aa.52964      xx.xx.xx.bb.6333     CLOSE_WAIT
tcp4       0      0  xx.xx.xx.aa.52961      xx.xx.xx.bb.6333     CLOSE_WAIT

• Recall TCP four waving
  ESTABLISHED indicates that the connection has been established
  CLOSE_WAIT represents the remote calculator close the connection, close the socket connection is waiting
  and phenomena in line with
• redis connection pool configuration
  according to the above netstat -ancan be basically determine the size of the connection pool is redis 8; binding code configured to, if not specified, the default is indeed 8

redis.clients.jedis.JedisPoolConfig
private int maxTotal = 8;
private int maxIdle = 8;
private int minIdle = 0;

• How to configure a larger pool of connections it?
  A. original configuration

@Bean
public RedisConnectionFactory redisConnectionFactory() {
    RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration();
    redisStandaloneConfiguration.setHostName(redisHost);
    redisStandaloneConfiguration.setPort(redisPort);
    redisStandaloneConfiguration.setPassword(RedisPassword.of(redisPasswd));
    JedisConnectionFactory cf = new JedisConnectionFactory(redisStandaloneConfiguration);
    cf.afterPropertiesSet();
    return cf;
}

readTimeout, connectTimeout not specified, there is a default value 2000 ms

org.springframework.data.redis.connection.jedis.JedisConnectionFactory.MutableJedisClientConfiguration
private Duration readTimeout = Duration.ofMillis(Protocol.DEFAULT_TIMEOUT);
private Duration connectTimeout = Duration.ofMillis(Protocol.DEFAULT_TIMEOUT); 

After modifying the configuration B.

• 1. A configuration: a portion of the interface has Deprecated

@Bean
public RedisConnectionFactory redisConnectionFactory() {
    JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
    jedisPoolConfig.setMaxTotal(16); // --最多可以建立16个连接了
    jedisPoolConfig.setMaxWaitMillis(10000); // --10s获取不到连接池的连接，
                                             // --直接报错Could not get a resource from the pool

    jedisPoolConfig.setMaxIdle(16);
    jedisPoolConfig.setMinIdle(0);

    JedisConnectionFactory cf = new JedisConnectionFactory(jedisPoolConfig);
    cf.setHostName(redisHost); // -- @Deprecated 
    cf.setPort(redisPort); // -- @Deprecated 
    cf.setPassword(redisPasswd); // -- @Deprecated 
    cf.setTimeout(30000); // -- @Deprecated 貌似没生效，30s超时，没有关闭连接池的连接；
                          // --redis没有设置超时，会一直ESTABLISHED；redis设置了超时，且超时之后，会一直CLOSE_WAIT

    cf.afterPropertiesSet();
    return cf;
}

• 1. Configuring way: This is the new way to configure the group to find a friend, the same effect

RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration();
redisStandaloneConfiguration.setHostName(redisHost);
redisStandaloneConfiguration.setPort(redisPort);
redisStandaloneConfiguration.setPassword(RedisPassword.of(redisPasswd));

JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
jedisPoolConfig.setMaxTotal(16);
jedisPoolConfig.setMaxWaitMillis(10000);
jedisPoolConfig.setMaxIdle(16);
jedisPoolConfig.setMinIdle(0);

cf = new JedisConnectionFactory(redisStandaloneConfiguration, JedisClientConfiguration.builder()
        .readTimeout(Duration.ofSeconds(30))
        .connectTimeout(Duration.ofSeconds(30))
        .usePooling().poolConfig(jedisPoolConfig).build());

reference

redistemplate- cursor scan Precautions

How to use RedisTemplate access Redis data structures

Redis is used in the Keys and Scan

Redis-depth understanding of the scan command

spring-boot-starter-redis Detailed Configuration

CLOSE_WAIT reason a large number of online investigation

How to configure standAlone redis version of jedisPool

Jedis use a non-standard, leading to a substantial increase in the redis client close_wait bug