I. Introduction
When using JedisPoll multi-threaded writing, the broken pipe is reported periodically, and the task is normal after restarting, and the error occurs again after a period of time.
JedisPool is configured as follows:
val config = new JedisPoolConfig
config.setMaxIdle(20)
config.setMinIdle(20)
config.setNumTestsPerEvictionRun(-2)
config.setTimeBetweenEvictionRunsMillis(30000)
config.setSoftMinEvictableIdleTimeMillis(3600000)
config.setMinEvictableIdleTimeMillis(-1)
config.setTestOnBorrow(false)
config.setTestOnReturn(false)
config.setTestWhileIdle(false)2. Problem analysis and solution
1. Problem scenario
The task is very simple. JedisPool is used to write data in multiple threads in the cluster. The error will be thrown after the task runs for a period of time. After a try catch, it is found that the task is an occasional error, and not all writing fails. The error is reported. Preliminary analysis is that the getResource method may obtain invalid connections, resulting in occasional write failures, and the task runs normally after restarting because the redis connections acquired by the task just start are valid, so there is no Broken pipe.
2. Problem solving
Through the configuration, you can see that the following three parameter settings are all false:
config.setTestOnBorrow(false)
config.setTestOnReturn(false)
config.setTestWhileIdle(false)Set TestOnBorrow to true or testWhileIdle to true, if that doesn't work, set both to true.
3. Parameter meaning
· setTestOnBorrow
borrow is to obtain the connection from the JedisPool connection pool. This parameter controls the validity of the connection when obtaining the redis connection. If it is checked that the link is invalid, it will be cleaned up and re-acquired for a new connection. Here you can refer to the getResource method in the source code , this method executes the borrowObject method from the Pool to obtain the connection. The location of this class is redis.client.util.Pool, and the underlying implementation refers to org.apache.commons.pool2.impl.GenericObjectPool.
public T getResource() {
try {
return internalPool.borrowObject();
} catch (Exception e) {
throw new JedisConnectionException("Could not get a resource from the pool", e);
}
}After configuring this parameter, when borrow calls to obtain a new connection, if the connection fails, the activeObject method will be called to reset the internal state of the connection, which is equivalent to obtaining a new connection:
try {
this.factory.activateObject(p);
} catch (Exception var15) {
try {
this.destroy(p);
} catch (Exception var14) {
}
p = null;
if (create) {
NoSuchElementException nsee = new NoSuchElementException("Unable to activate object");
nsee.initCause(var15);
throw nsee;
}
}
· SetTestOnReturn
return is a parameter configured when calling returnResource. This parameter controls the validity of the connection when the resource connection is returned to the connection pool. If the connection is invalid, the connection will be cleaned up. This method executes the returnResourceObject method from the Pool to return the connection. The location of this class is redis.client.util.pool, and the underlying implementation also refers to GenericObjectPool.
public void returnResourceObject(final T resource) {
if (resource == null) {
return;
}
try {
internalPool.returnObject(resource);
} catch (Exception e) {
throw new JedisException("Could not return the resource to the pool", e);
}
}If this parameter is configured, the code will call the ValidateObject method on the connection to judge the validity when returningObjectResource, and if invalid, destroy:
if (this.getTestOnReturn() && !this.factory.validateObject(p)) {
try {
this.destroy(p);
} catch (Exception var10) {
this.swallowException(var10);
}
try {
this.ensureIdle(1, false);
} catch (Exception var9) {
this.swallowException(var9);
}
this.updateStatsReturn(activeTime);
}· SetTestWhileIdle
This parameter controls testing for idle connections. This parameter corresponds to config.setTimeBetweenEvictionRunsMillis(30000), where the idle judgment is based on the millisecond time set in this parameter. The validity of the idle connection is checked here, and the evict method is what actually performs the cleanup:
public void evict() throws Exception {
this.assertOpen();
if (this.idleObjects.size() > 0) {
PooledObject<T> underTest = null;
EvictionPolicy<T> evictionPolicy = this.getEvictionPolicy();
synchronized(this.evictionLock) {
EvictionConfig evictionConfig = new EvictionConfig(this.getMinEvictableIdleTimeMillis(), this.getSoftMinEvictableIdleTimeMillis(), this.getMinIdle());
boolean testWhileIdle = this.getTestWhileIdle();
int i = 0;
int m = this.getNumTests();
while(true) ...Line 7 here also involves two time parameters, MinEvictableIdleTimeMillis and SoftMinEvictableIdleTimeMillis. This involves the selection of thread pool idleEvictTime. The former is read first. If the former is negative, the default is maxLong. If the former is not configured, the latter is read, so the thread pool Some configuration parameters of -1, -2 will appear, and there is actually no time of -1 ms.
public EvictionConfig(long poolIdleEvictTime, long poolIdleSoftEvictTime, int minIdle) {
if (poolIdleEvictTime > 0L) {
this.idleEvictTime = poolIdleEvictTime;
} else {
this.idleEvictTime = 9223372036854775807L;
}
if (poolIdleSoftEvictTime > 0L) {
this.idleSoftEvictTime = poolIdleSoftEvictTime;
} else {
this.idleSoftEvictTime = 9223372036854775807L;
}
this.minIdle = minIdle;
}The real judgment to start executing the evict logic is in the following function:
public boolean evict(EvictionConfig config, PooledObject<T> underTest, int idleCount) {
return config.getIdleSoftEvictTime() < underTest.getIdleTimeMillis() && config.getMinIdle() < idleCount || config.getIdleEvictTime() < underTest.getIdleTimeMillis();
}Eviction is performed if any of the following conditions are met:
A. The connection idle time is greater than softMinEvictableIdleTimeMillis and the idle number of the connection pool is less than minIdleNum, which is the third parameter in EvictionConfig
B. The connection idle time is greater than minEvictableIdleTimeMillis
Of course, it will not only be evicted. To a certain extent, the thread pool will also perform ensureIdle to create a new connection to ensure that there are enough connections available in the thread pool.
3. Summary
The above error report and scene setting TestOnBorrow = true and TestWhileIdle = true are perfectly solved, but it will also bring a certain performance loss. You can adjust it according to your own scene. In general, you can try to only turn on TestOnBorrow.