Deadlock online troubleshooting

Problem Description:
Online service suddenly hung up, can not be called to see if the service logs found Dubbo thread pool all full:

2019-11-22 14:35:26.271  WARN 26516 --- [New I/O server worker #1-2] c.a.d.c.t.support.AbortPolicyWithReport  :  [DUBBO] Thread pool is EXHAUSTED! Thread Name: DubboServerHandler-192.168.10.26:12350, Pool Size: 200 (active: 200, core: 200, max: 200, largest: 200), Task: 6786 (completed: 6586), Executor status:(isShutdown:false, isTerminated:false, isTerminating:false), in dubbo://192.168.10.26:12350!, dubbo version: 2.6.2, current host: 192.168.10.26

Not much traffic, but the thread has soared, speculation may be where there is a cycle of death or where a deadlock occurred.

First, detect what CPU usage server found in the normal range, basically ruled out where there is a cycle of death.

First find out the process of the service, use jstackthe command dumpthread analysis.

"DubboServerHandler-192.168.10.26:12350-thread-200" #240 daemon prio=5 os_prio=0 tid=0x00007ffa7c141800 nid=0x6c89 waiting on condition [0x00007ffa17c7c000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000000e0d24020> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:590)
    at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:441)
    at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:362)
    at redis.clients.util.Pool.getResource(Pool.java:49)
    at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226)
    at (RedisHelper.java:322)
    at (RedisHelper.java:106)
    at BlockResourceManager.java:54)
    - locked <0x00000000e0ec22d0> 

You can see the last thread locked <0x00000000e0ec22d0>, that it holds with the lock, while other states are threads waiting to lock <0x00000000e0ec22d0>.
It seems that this thread has led to other threads are blocked.

So what is the cause of this thread can not quick-release lock it?
Continue to look at the above information, it should be from the JedisPoolacquisition of JedisClient, but the client has not been available in the pool end, it is blocked.

The last reason is a method used up a client does not return to the pool, resulting in client resources are exhausted.