Didi side: How to set the priority of thread pool tasks?

Said it in front

In the reader exchange group (50+) of Nien, a 40-year-old architect , some friends have recently obtained interview qualifications from first-tier Internet companies such as Didi, Jitu, Youzan, Xiyin, Baidu, and NetEase, and met many Very important interview questions:

How to design a thread pool?

Please write a simple thread pool by hand?

Just yesterday, a friend was interviewing Didi and encountered a series of questions related to the underlying principles of the thread pool. He failed to answer the question well, causing the interview to fail.

The real interview question of Didi that my friend encountered was this series of questions related to the underlying principle of the thread pool. Let’s use the boy’s original words.

The original words of my friend are as follows:

Brother Eng, Didi recently encountered a question and asked:

How thread scheduling is performed at the bottom of the thread pool?
How threads perform preemption and priority settings,
How to set up a thread pool in scenarios with priority requirements. I haven't found the answer online.

Nien’s tip: Thread pool knowledge is both the core knowledge for interviews and the core knowledge for development . Therefore, here Nien will give you a systematic and systematic thread pool sorting, so that you can fully demonstrate your strong "technical muscles" and make the interviewer "can't help himself and drool" .

This question and reference answers are also included in the V110 version of our " Nien Java Interview Guide PDF " for reference by subsequent friends to improve everyone's 3-level architecture, design, and development levels.

Please follow this public account [Technical Freedom Circle] to get the PDFs of "Nien Architecture Notes", "Nien High Concurrency Trilogy" and "Nien Java Interview Guide"

Article directory

Analysis of Didi’s real interview questions

First question: How is thread scheduling performed at the bottom of the thread pool?

This is a basic question. For specific answers, please refer to Nien's " Java High Concurrency Core Programming Volume 1 Enhanced Edition "

Second question: How to preempt and prioritize threads?

This requires everyone to understand the operating principle of the thread pool. It is best to write a simple thread pool by yourself to deepen your impression.

How to write a thread pool by hand? Please refer to the article by the Nien Architecture Team

NetEase: How to design a thread pool? Please write a simple thread pool by hand?

Third question: How to set up the thread pool in scenarios with priority requirements?

This article will be used to answer this question.

Basic principles of thread pool

Generally speaking, everyone uses thread pools for concurrent execution and concurrent scheduling of tasks, and uses the pooled architecture to save performance losses caused by thread creation and destruction.

By default, with the thread pool, everyone submits tasks to the thread pool and the thread pool schedules them.

Tasks submitted to the thread pool are scheduled according to the thread pool's scheduling rules. The scheduling rules of the thread pool are roughly as follows:

NOTE: Please click on the image for a clear view!

If the core threads of the thread pool are very busy, tasks need to be queued and entered into the internal work queue of the thread pool, as shown roughly in the following figure:

NOTE: Please click on the image for a clear view!

After the worker thread completes the task at hand, it will repeatedly obtain tasks from the internal work queue (such as LinkedBlockingQueue) in an infinite loop for execution.

How to set up a thread pool in scenarios with priority requirements?

In an ordinary thread pool, there is no priority privilege between tasks. It can be understood as a first-in, first-out fair scheduling mode.

Sometimes, tasks have priority privileges. Instead of scheduling according to first-in-first-out, they need to be scheduled according to priority.

So, if there are some priority changes between different tasks, what should be done?

The solution is very simple, which is to replace the work queue in the thread pool and use a priority unbounded blocking queue to manage asynchronous tasks.

First, let’s take a look at several typical work queues

ArrayBlockingQueue : A bounded blocking queue implemented using an array, with first-in first-out characteristics
LinkedBlockingQueue : A blocking queue implemented using a linked list. The feature is first in, first out. You can set its capacity. The default is Interger.MAX_VALUE. The feature is first in first out.
PriorityBlockingQueue : an unbounded blocking queue with priority implemented using a balanced binary tree heap
DelayQueue : Unbounded blocking delay queue. Each element in the queue has an expiration time. When elements are obtained from the queue, only expired elements will be dequeued. The head element of the queue is the last element to expire.
SynchronousQueue : A blocking queue that does not store elements. Each insertion operation must wait until another thread calls the removal operation, otherwise the insertion operation will remain blocked.

Use the priority unbounded blocking queue PriorityBlockingQueue instead of the queue without priority ArrayBlockingQueue or LinkedBlockingQueue.

As an extra reminder, if it is an ordinary task with no priority, in general, it is recommended that you refer to the source code of rocketmq and use a bounded LinkedBlockingQueue as the task queue.

The source code of rocketmq uses a large number of thread pools, as shown below:

These task queues use bounded LinkedBlockingQueue, as shown below:

If tasks have priorities, task queues need to be introduced and managed.

At this time, you need to use the priority unbounded blocking queue PriorityBlockingQueue. The following is an example.

Design and implementation of priority task thread pool

There are two key points in implementing a priority task thread pool:

Use PriorityBlockingQueue as the task queue for the thread pool.
Submitted tasks have the ability to be sorted.

Using PriorityBlockingQueue as the task queue for the thread pool

Still constructing the thread pool based on ThreadPoolExecutor, we know that the constructor of ThreadPoolExecutor has a workQueue parameter, where the PriorityBlockingQueue priority queue can be passed in.

In the buildWorkQueue() method, construct one PriorityBlockingQueue<E>, and its constructor can pass in a comparator to meet the requirements.

There are two main points here:

Replace the default blocking queue of the thread pool with PriorityBlockingQueue, and the corresponding incoming thread class needs to be implemented for Comparable<T>comparison.
The data structure of PriorityBlockingQueue determines that tasks with the same priority cannot guarantee FIFO and need to control the order by themselves.

Submitted tasks have the ability to be sorted

The input parameters of ThreadPoolExecutor's submit, invokeXxx, and execute methods are Runnable and Callable, and they do not have sortable attributes.

We can make an implementation class PriorityTask and add some additional attributes to give them sorting capabilities.

Test the custom priority thread pool

Submit tasks with three different priorities for testing

Submit those with higher priority later.
Submit earlier with lower priority

Execute the one with higher priority first

Those with lower priority will be executed later.

Issues with PriorityBlockingQueue queue

Mainly, PriorityBlockingQueue is unbounded and its offer method always returns true.

This will bring about two problems:

First, OOM risk;

Second, the maximum number of threads is invalid

Third, the rejection strategy fails

How to solve it?

Method 1 : You can inherit PriorityBlockingQueue and rewrite the offer method of this class. If the elements exceed the specified number, it will directly return false, otherwise the original logic will be called.

Method 2 : Expand the submit, invokeXxx, and execute methods of the thread pool to count, check, and limit the number of tasks inside.

It is recommended that everyone use method one first.

Let’s talk about the heap structure of PriorityQueue

At this point in the interview, it is easy to have problems with the heap structure of PriorityQueue.

Because, PriorityBlockingQueue is the blocking version of PriorityQueue

PriorityQueue is a heap implementation provided by Java

PriorityQueue is a minimum heap by default . If you call the constructor using the maximum heap, you need to pass in Comparatorthe comparison sorting rules.

// 构造小顶堆
PriorityQueue<Integer> priorityQueue = new PriorityQueue<>((o1, o2) -> o1 - o2);

// 构造大顶堆
PriorityQueue<Integer> priorityQueue = new PriorityQueue<>((o1, o2) -> o2 - o1);

PriorityQueueThe interface is implemented Queue, and its commonly used functions are shown in the table

This is why the heap is called a priority queue

operate	Throw exception	Don't throw exception
Insert new element	add(e)	offer(e)
Delete the top element of the heap	remove	poll
Return the top element of the heap	element	peek

Although PriorityQueue in Java implements the Queue interface, it is not a queue, nor does it delete elements in the order of "first in, first out".

The order in which PriorityQueue is removed has nothing to do with the order in which elements were added. PriorityQueue operates elements in the order of the minimum heap.

Therefore, PriorityQueue is a heap, and each time the function remove or poll is called, the element at the top of the heap will be deleted.

In the same way, the functions element and peek of PriorityQueue both return the element at the top of the heap, that is, the element with the largest or smallest value is returned according to the type of the heap, regardless of the order in which the elements are added.

Coming to the basics of algorithms, what is a heap?

A heap (also called a priority queue) is a special tree-shaped data structure.

According to the relationship between the value of the root node and the value of the child node, the heap is divided into a maximum heap and a minimum heap.

In a max-heap, the value of each node is always greater than or equal to the value of any of its child nodes, so the root node of the max-heap is the maximum value of the entire heap .
In a min-heap, the value of each node is always less than or equal to the value of any of its child nodes, so the root node of the min-heap is the minimum value of the entire heap.

For example, Figure (a) shows a max-heap, and Figure (b) shows a min-heap.

Heaps are usually implemented as complete binary trees . In a complete binary tree, all levels except the lowest level are filled with nodes, where nodes are inserted from left to right whenever possible. Both heaps in the picture above are complete binary trees.

A complete binary tree can be implemented with an array, so a heap can also be implemented with an array . If you start from the root node of the heap and traverse it layer by layer from top to bottom, and number each node in each layer from left to right in the order of 0, 1, 2, etc., put the node numbered 0 into the array, and the subscript is At position 0, the node numbered 1 is put into the array at position 1, and by analogy, all nodes in the heap can be added to the array. The heap in (a) above can be represented by an array in the form shown in (a) below, and the heap in (b) above can be represented by an array in the form shown in (b) below.

If the subscript of an element in the array is i, then the subscript of the parent node of its corresponding node in the heap is in the array (i - 1) / 2, and the subscripts of its left and right child nodes in the array are respectively 2 * i + 1and 2 * i + 2.

Add element to heap

In order to add a new node to the max heap, there are three steps:

First find the first vacant position from top to bottom and left to right, and add the new node to the vacant position.
If the value of the new node is greater than the value of its parent, swap it with its parent.
This swapping process is repeated until the new node's value is less than or equal to its parent node, or it has reached the top position of the heap.

The process of adding a new node to a min-heap is similar, the only difference is to ensure that the value of the new node is greater than or equal to its parent node.

Therefore, the addition operation of the heap is a bottom-up operation.

For example, if a new element 95 is added to the max heap in (a) above:

Since the right child node of node 60 is the first vacant position, create a new node 95 and make it the right child node of node 60
At this time, the value of the new node 95 is greater than the value of its parent node 60, which violates the definition of the maximum heap, so it and its parent node are exchanged
Since the value of new node 95 is still greater than the value of its parent node 90, new node 95 and its parent node 90 are exchanged. At this point the heap has met the definition of the maximum heap.

The overall process is as follows:

The insertion operation of the heap may require exchanging nodes in order to put the nodes in the appropriate position. The number of exchanges is at most the depth of the binary tree. Therefore, if there are n nodes in the heap, the time complexity of its insertion operation is O(logn).

Remove element from heap

Usually only the element at the top of the heap is removed .

Take deleting the top node of the max heap as an example:

Move the lowest rightmost node of the heap to the top of the heap
If the value of its left child node or right child node is greater than it at this time, then it is exchanged with the node with a larger value among the left and right child nodes.
If the value of the node is still less than the value of its child nodes after the exchange, it is exchanged again until the value of the node is greater than or equal to the value of its left and right child nodes, or the lowest level is reached.

The process of deleting the top node of the min-heap is similar to this, the only difference is to ensure that the value of the node is smaller than the value of its left and right children.

Therefore, the heap deletion operation is a top-down operation.

For example, after deleting the top element of the max-heap in (a) above:

Move the rightmost node 60 at the lowest level to the top of the maximum heap, as shown in (c) below
At this time, node 60 has a smaller value than its left child node 80 and right child node 90, so it is exchanged with the right child node 90 that has a larger value. The heap after the exchange is as shown in Figure (d)
At this time, node 60 is greater than its left child node 30, which meets the definition of the maximum heap.

The deletion operation of the heap may require exchanging nodes in order to put the nodes in the appropriate position. The number of exchanges is at most the depth of the binary tree. Therefore, if there are n nodes in the heap, the time complexity of its deletion operation is O(logn).

Say it at the end

Thread pool interview questions are very common interview questions.

If everyone can answer the above content fluently and thoroughly, the interviewer will basically be shocked and attracted by you.

Before the interview, it is recommended that you systematically review the 5,000-page " Nien Java Interview Guide PDF ". If you have any questions during the question review process, you can come to talk to Nien, a 40-year-old architect.

In the end, the interviewer loved it so much that he "can't help himself and his mouth watered" . The offer is coming.