Common JedisConnectionException analysis

Recently, Redis was used in project development, and the java client Jedis recommended by the official website was selected.
Redis common command learning: http://redis.io/commands
Redis officially recommends Java client Jedis (including the implementation of all Redis commands): https://github.com/xetorthio/jedis

The most common exception in the use of Jedis JedisConnectionException Sometimes it does bring us a lot of confusion, this exception usually occurs in two scenarios.

1. When we execute the getResource() of the following JedisPool class instance, a can't get a resource exception is thrown.

The exception code is as follows:

Redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool

at Redis.clients.util.Pool.getResource(Pool.Java:22)

Analysis:

redis.clients.util.Pool.getResource An available redis connection will be returned from the pool of JedisPool instances. Analysis of the source code shows that JedisPool extends redis.clients.util.Pool<Jedis> . And Pool<T> is

used to manage Jedis instances through org.apache.commons.pool.impl.GenericObjectPool in the commons-pool open source toolkit . So we may find the answer by analyzing GenericObjectPool.
First look at common-pool's api: http://commons.apache.org/pool/apidocs/index.html?org/apache/commons/pool/impl/GenericObjectPool.html.
Three important properties are:
MaxActive: the maximum number of available connection instances, there is no limit when it is a negative value.
MaxIdle: The maximum number of idle connection instances, there is no limit when it is negative. An instance of Idle is usually made available via the activateObject() method of org.apache.commons.pool.BasePoolableObjectFactory<T> before it can be used.
MaxWait: The maximum number of connections to wait for available, in milliseconds (million seconds).
     (Note: The borrowObject() method of the GenericObjectPool class actually called by the pool.getResource() method will block and wait until there is no available connection (idle/active) according to the value of the MaxWait variable. Please refer to the api for the specific meaning.

) When there is no active/idle connection in the connection pool, it will wait for the maxWait time. If there is no available connection after the timeout, a Could not get a resource from the pool exception will be thrown. Therefore, in order to avoid such errors,

we should reasonably set the values ​​of these three parameters according to the actual situation of the program. At the same time, we should also handle this exception reasonably in the program method of obtaining a connection. When there is no connection available, wait for a period of time before Getting it might be a better option.


2. When we operate redis after obtaining the connection, redis.clients.jedis.exceptions.JedisConnectionException: Java.NET.SocketTimeoutException: Read timed out exception is thrown.

The exception code is as follows:

redis.clients.jedis.exceptions.JedisConnectionException: java.Net.SocketTimeoutException: Read timed out

at redis.clients.jedis.Protocol.process(Protocol.java:79)
at redis.clients.jedis.Protocol.read (Protocol.java:131)
at redis.clients.jedis.Connection.getIntegerReply(Connection.java:188)
at redis.clients.jedis.Jedis.sismember(Jedis.java:1266
) time of my day. We all know that Redis operates on memory, and the speed should be in milliseconds. This is our usual understanding, so when there is a timeout of several seconds for Redis operations, can you imagine it?
Let's analyze the source code of Jedis first, taking the sadd operation as an example:

    public Long sadd(final String key, final String... members) {
    checkIsInMulti();
    client.sadd(key, members);
    return client.getIntegerReply();
    }

client is an instance of redis.clients.jedis.Client.java, the inheritance relationship is as follows:
public class Client extends BinaryClient implements Commands;

public class BinaryClient extends Connection;
Connection It wraps the socket operation on the Redis server. The command write operation sends the command information to the redis server through the socket.getOutputStream() output stream. After the command is written, the
command , there must be a delay time between command execution and result return, which is the time it takes for a Jedis to call a redis command operation.

It should be noted that the Redis server executes all commands sent by the connection in a single thread, that is to say, no matter how many clients are sending commands concurrently, the redis-server side is processed in a single thread and processed in the default FIFO mode. request,

this can be configured in the redis.conf configuration file. For the detailed operation mechanism of redis server, see: http://redis.io/documentation
So client.sadd(key, members); just sends the command information to the redis server after the call is completed. Whether it is executed depends on the load of the redis server. Then, through client.getIntegerReply(); wait (time out) to return the result.
Connection has multiple options when initializing a socket. The methods for setting socket time out are as follows:

    public void rollbackTimeout() {
             try {
               

    socket.setSoTimeout(timeout);
               

    socket.setKeepAlive(false);
             } catch (SocketException ex) {
              

     throw new JedisException (ex);
             }
         }

From redis.clients.jedis.Protocol.DEFAULT_TIMEOUT = 2000 We know that the default timeout is 2 seconds, which is already very long compared to the millisecond speed of redis operation memory, so why do we still Encountered
ava.net.SocketTimeoutException: Read timed out exception? Although redis operates on the average millisecond level, it may not be so fast when the amount of data is large. In my development process, I encountered a collection to

With a data volume of tens of millions, it is normal for an operation to have a timeout in seconds, and this is already the case when the machine performance is very good, not to mention that the machine we developed locally will be slower than the production server. Therefore, when initializing JedisPool, the connection pool parameters should be set reasonably through redis.clients.jedis.JedisPoolConfig according to the actual

situation , and the timeout time for socket to read the input InputStream should be reasonably set through the edisPool construction method.

    pool = new JedisPool(config, host, port, 100000);

Note that the fourth parameter time out is set to the timeout we can tolerate, in milliseconds. But don't know why the parameter type is int instead of long since the unit is milliseconds.

After setting the fourth parameter, I operated on the 40 million data set with a timeout of about 5 seconds at most once, and the problem was basically solved.