JedisPool的getResource()方法配置不当导致服务假死

JedisPool的getResource()方法配置不当导致服务假死

dubbo服务中使用jedis，在从JedisPool获取jedis时超时导致dubbo服务假死

"DubboServerHandler-10.0.101.208:20880-thread-22" daemon prio=10 tid=0x00007f52f00b7800 nid=0x3d85 waiting on condition [0x00007f52ea3a1000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00001100c65602d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:524)
	at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:433)
	at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:360)
	at redis.clients.util.Pool.getResource(Pool.java:48)
	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:88)
	at com.alibaba.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.handleRequest(HeaderExchangeHandler.java:84)
	at com.alibaba.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.received(HeaderExchangeHandler.java:170)
	at com.alibaba.dubbo.remoting.transport.DecodeHandler.received(DecodeHandler.java:52)
	at com.alibaba.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run(ChannelEventRunnable.java:82)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

JedisPool.java:

@Override
public Jedis getResource() {
  Jedis jedis = super.getResource();
  jedis.setDataSource(this);
  return jedis;
}


public T getResource() {
  try {
    return internalPool.borrowObject();
  } catch (Exception e) {
    throw new JedisConnectionException("Could not get a resource from the pool", e);
  }
}



public T borrowObject(long borrowMaxWaitMillis) throws Exception {
    assertOpen();

    AbandonedConfig ac = this.abandonedConfig;
    if (ac != null && ac.getRemoveAbandonedOnBorrow() &&
            (getNumIdle() < 2) &&
            (getNumActive() > getMaxTotal() - 3) ) {
        removeAbandoned(ac);
    }

    PooledObject<T> p = null;

    // Get local copy of current config so it is consistent for entire
    // method execution
    boolean blockWhenExhausted = getBlockWhenExhausted();

    boolean create;
    long waitTime = 0;

    while (p == null) {
        create = false;
        if (blockWhenExhausted) {
            p = idleObjects.pollFirst();
            if (p == null) {
                create = true;
                p = create();
            }
            if (p == null) {
                if (borrowMaxWaitMillis < 0) {
                    p = idleObjects.takeFirst();
                } else {
                    waitTime = System.currentTimeMillis();
                    p = idleObjects.pollFirst(borrowMaxWaitMillis,
                            TimeUnit.MILLISECONDS);
                    waitTime = System.currentTimeMillis() - waitTime;
                }
            }
            if (p == null) {
                throw new NoSuchElementException(
                        "Timeout waiting for idle object");
            }
            if (!p.allocate()) {
                p = null;
            }
        } else {
            p = idleObjects.pollFirst();
            if (p == null) {
                create = true;
                p = create();
            }
            if (p == null) {
                throw new NoSuchElementException("Pool exhausted");
            }
            if (!p.allocate()) {
                p = null;
            }
        }

        if (p != null) {
            try {
                factory.activateObject(p);
            } catch (Exception e) {
                try {
                    destroy(p);
                } catch (Exception e1) {
                    // Ignore - activation failure is more important
                }
                p = null;
                if (create) {
                    NoSuchElementException nsee = new NoSuchElementException(
                            "Unable to activate object");
                    nsee.initCause(e);
                    throw nsee;
                }
            }
            if (p != null && getTestOnBorrow()) {
                boolean validate = false;
                Throwable validationThrowable = null;
                try {
                    validate = factory.validateObject(p);
                } catch (Throwable t) {
                    PoolUtils.checkRethrow(t);
                    validationThrowable = t;
                }
                if (!validate) {
                    try {
                        destroy(p);
                        destroyedByBorrowValidationCount.incrementAndGet();
                    } catch (Exception e) {
                        // Ignore - validation failure is more important
                    }
                    p = null;
                    if (create) {
                        NoSuchElementException nsee = new NoSuchElementException(
                                "Unable to validate object");
                        nsee.initCause(validationThrowable);
                        throw nsee;
                    }
                }
            }
        }
    }

    updateStatsBorrow(p, waitTime);

    return p.getObject();
}

修改JedisPoolConfigBean配置:

<bean id="jedisPoolConfig" class="redis.clients.jedis.JedisPoolConfig">
	<property name="maxTotal" value="50" />
	<property name="maxIdle" value="10" />
	<property name="maxWaitMillis" value="1000" />
	<property name="testOnBorrow" value="true" />
</bean>

maxWaitMillis默认是-1,表示如果从jedispool中获取jedis超时后会一直阻塞;而将maxWaitMillis设置成一个正数，表示在规定时间超时后会抛出异常