How to handle when the thread pool is full and spring parameter settings:

How to handle when the thread pool is full and spring parameter settings:

 

 

The problem of multithreading, not only consider using thread pools in the code, but maybe configure it on spring, etc.

The concurrency problem is not, simply synchronization in the code, database zk, etc., not only in nginx, tomcat tuning is also set in the jvm database.

 

 

Introduction to thread pools

Since JDK1.5, Java 's concurrent package has provided thread pool java.util.concurrent.ThreadPoolExecutor , let's take a look at the meaning of each field: 
corePoolSize The number of core threads, which refers to the reserved thread pool size (not exceeding the maximumPoolSize value) , there are at most corePoolSize threads working in the thread pool). 
maximumPoolSize refers to the maximum size of the thread pool (maximum corePoolSize threads can run in the thread pool). 
keepAliveTime refers to the timeout for the end of idle threads (when a thread is not working, keepAliveTime will stop the thread for a long time). 
unit is an enumeration representing the unit of keepAliveTime (with NANOSECONDS, MICROSECONDS, MILLISECONDS, SECONDS, MINUTES, HOURS, DAYS, 7 optional values). 
workQueue represents a queue for storing tasks (a queue for threads that need to be executed by the thread pool). 
handler Rejection policy (how to handle adding task if it fails).

Regarding the relationship between corePoolSize, maxPoolSize, and queueCapacity: corePoolSize is the initial number of threads. When the threads of corePoolSize are all executing, the Runnable is temporarily placed in the queueCapacity buffer queue to wait. When the queueCapacity is full, the Runnable will be placed The number of threads is expanded from corePoolSize to maxPoolSize. If the queueCapacity cache queue is still full at this time, the subsequent Runnable objects will be discarded by abort when they are added .

This brother provides a good example of code description, please refer to: http://dmwdmc.iteye.com/blog/1882475 . In the example, there is a scenario in which the entire thread pool is simulated and the subsequent tasks are abandoned.

Thread pool pit 1

Note: The following thread pool configurations are all thread pools using spring encapsulation. 
The scene of the pit: An interface needs to access the database four times , and there is no necessary connection between the four results. Why not put the four database access operations into four threads to access at the same time, thus greatly saving time . 
OK, the landlord did just that, and the four threads were thrown into a unified thread pool for execution. So here comes the pit. This interface often has "surprise" traffic at a fixed time of the day. On the second day of the launch, all six tomcats hang up. Check the heap dump file and find that there is an OOM error, and the six tomcats are almost the same. Appears around the same time. With a guilty conscience, the landlord thought of the thread pool he added. The following is the thread pool configuration of the landlord, everyone's appreciation (the configuration is copied directly from the configuration of other people, without thinking too much):

    <bean id="resQueryBaseInfoExecutor"
          class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
        <property name="threadNamePrefix" value="resQueryBaseInfoExecutor-" />
        <property name="corePoolSize" value="5" />
        <property name="keepAliveSeconds" value="100" />
        <property name="maxPoolSize" value="10" />
        <property name="queueCapacity" value="100000" />
    </bean>

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8

The queueCapacity is set to 100000, the purpose is to ensure that all tasks are not accepted and will not be abandoned because the thread pool is full, but obviously when the cache queue really reaches 10W, OOM should not be far away... The 
landlord thought deeply about the reason : The corePoolSize is too small, resulting in the thread pool throughput being too small, and the queueCapacity configuration is too large, coupled with poor throughput capacity, which directly leads to the waiting queue reaching 10W objects, causing the JVM to OOM smoothly, causing the entire application to hang, resulting in downtime serious problem with the machine.

Thread pool pit 2

The corePoolSize configuration is small, and the queueCapacity configuration is also small, resulting in poor thread pool throughput and a small number of caches. It is very likely that the following threads will be rejected after the traffic comes up a little, resulting in loss of data operations. This is also what the landlord personally experienced:

    <bean id="resQueryBaseInfoExecutor"
          class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
        <property name="threadNamePrefix" value="resQueryBaseInfoExecutor-" />
        <property name="corePoolSize" value="5" />
        <property name="keepAliveSeconds" value="100" />
        <property name="maxPoolSize" value="10" />
        <property name="queueCapacity" value="500" />
    </bean>

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8

Therefore, in summary, the three parameters of corePoolSize, maxPoolSize, and queueCapacity can be set correctly only by correctly estimating the traffic of the interface and the approximate execution time of each thread task, and calculating the throughput of each thread.