This series of related blog, Mu class reference column Java source code and system manufacturers interviewer succinctly Zhenti
below this column is GitHub address:
Source resolved: https://github.com/luanqiu/java8
article Demo: HTTPS: // GitHub. com / luanqiu / java8_demo
classmates can look at it if necessary)
Java source code analysis and interview questions-DelayQueue source code analysis
Introductory Remarks
The blocking queue we mentioned earlier is executed immediately when there are sufficient resources. The queue we talked about in this chapter is special, it is a kind of delayed queue, which means delayed execution, and you can set how long to delay execution, for example, after 5 seconds, and then execute it. Reconciliation and so on.
1 Overall design
DelayQueue The bottom of the delay queue uses the ability to lock. For example, if you want to delay execution for 5 seconds backward at the current time, then the current thread will sleep for 5 seconds. When the thread is woken up after 5 seconds, if the resource can be obtained, the thread It can be executed immediately. It seems to be simple in principle, but the internal implementation is very complicated. There are many difficulties. For example, when there are insufficient running resources and multiple threads are woken up at the same time, how to queue up and wait? For example, when does it block? When does execution start, etc.? Next we look at how it is implemented from the perspective of source code.
1.1 Class notes
Class annotations are relatively simple, only three concepts are mentioned:
- The elements in the queue will be executed when they expire, the closer to the head of the queue, the sooner they will expire;
- Unexpired elements cannot be taken;
- Empty elements are not allowed.
These three concepts are actually three problems. Below we will take a look at how these three points are implemented.
1.2 Class diagram
The class diagram of DelayQueue is the same as the previous queue, not to mention, the key is that the DelayQueue class is generic, as follows:
public class DelayQueue<E extends Delayed> extends AbstractQueue<E>
implements BlockingQueue<E> {
It can be seen from the generics that the elements in DelayQueue must be a subclass of Delayed. Delayed is a key interface that expresses the ability to delay. It inherits the Comparable interface and defines how long the remaining time expires, as follows:
public interface Delayed extends Comparable<Delayed> {
long getDelay(TimeUnit unit);
}
In other words, the elements in the DelayQueue queue must implement the Delayed interface and the Comparable interface, and override the getDelay method and the compareTo method. Otherwise, when compiling, the compiler will remind us that the element must force the Delayed interface.
In addition, DelayQueue also uses a lot of functions of PriorityQueue queue, which is very similar to SynchronousQueue queue, a lot of logic of other basic classes is reused, the code example is as follows:
PriorityQueue Chinese is called priority queue, the role here is to Prioritize according to the expiration time, so that the first to expire can be executed first, used to achieve the first point in the class comment.
The idea of reuse here is quite important. We often encounter this idea in the source code, such as the ability of LinkedHashMap to reuse HashMap, the ability of Set to reuse Map, and the ability of DelayQueue to reuse PriorityQueue here. . To summarize, what do you need to do if you want to reuse:
- It is necessary to abstract the functions that can be reused as much as possible and open up extensible places. For example, in the method of operating arrays, HashMap opens many methods to LinkedHashMap after the beginning, which is convenient for LinkedHashMap to sort, delete, etc .;
- Use combination or inheritance to reuse, such as inheritance used by LinkedHashMap, combination used by Set and DelayQueue, the meaning of combination is to rely on reusable classes.
2 Demo
In order to facilitate everyone's understanding, I wrote a demo demo that demonstrates:
public class DelayQueueDemo {
// 队列消息的生产者
static class Product implements Runnable {
private final BlockingQueue queue;
public Product(BlockingQueue queue) {
this.queue = queue;
}
@Override
public void run() {
try {
log.info("begin put");
long beginTime = System.currentTimeMillis();
queue.put(new DelayedDTO(System.currentTimeMillis() + 2000L,beginTime));//延迟 2 秒执行
queue.put(new DelayedDTO(System.currentTimeMillis() + 5000L,beginTime));//延迟 5 秒执行
queue.put(new DelayedDTO(System.currentTimeMillis() + 1000L * 10,beginTime));//延迟 10 秒执行
log.info("end put");
} catch (InterruptedException e) {
log.error("" + e);
}
}
}
// 队列的消费者
static class Consumer implements Runnable {
private final BlockingQueue queue;
public Consumer(BlockingQueue queue) {
this.queue = queue;
}
@Override
public void run() {
try {
log.info("Consumer begin");
((DelayedDTO) queue.take()).run();
((DelayedDTO) queue.take()).run();
((DelayedDTO) queue.take()).run();
log.info("Consumer end");
} catch (InterruptedException e) {
log.error("" + e);
}
}
}
@Data
// 队列元素，实现了 Delayed 接口
static class DelayedDTO implements Delayed {
Long s;
Long beginTime;
public DelayedDTO(Long s,Long beginTime) {
this.s = s;
this.beginTime =beginTime;
}
@Override
public long getDelay(TimeUnit unit) {
return unit.convert(s - System.currentTimeMillis(), TimeUnit.MILLISECONDS);
}
@Override
public int compareTo(Delayed o) {
return (int) (this.getDelay(TimeUnit.MILLISECONDS) - o.getDelay(TimeUnit.MILLISECONDS));
}
public void run(){
log.info("现在已经过了{}秒钟",(System.currentTimeMillis() - beginTime)/1000);
}
}
// demo 运行入口
public static void main(String[] args) throws InterruptedException {
BlockingQueue q = new DelayQueue();
DelayQueueDemo.Product p = new DelayQueueDemo.Product(q);
DelayQueueDemo.Consumer c = new DelayQueueDemo.Consumer(q);
new Thread(c).start();
new Thread(p).start();
}
}
打印出来的结果如下：
06:57:50.544 [Thread-0] Consumer begin
06:57:50.544 [Thread-1] begin put
06:57:50.551 [Thread-1] end put
06:57:52.554 [Thread-0] 延迟了2秒钟才执行
06:57:55.555 [Thread-0] 延迟了5秒钟才执行
06:58:00.555 [Thread-0] 延迟了10秒钟才执行
06:58:00.556 [Thread-0] Consumer end
The purpose of writing this code is mainly to demonstrate the example of delayed execution. Our general idea is:
- The elements of the newly created queue, such as DelayedDTO, must implement the Delayed interface. In the getDelay method, we have implemented the method how long is left until the expiration time.
- Define the producers and consumers of the queue elements, corresponding to the Product and Consumer in the code.
- Initialize and manage producers and consumers, corresponding to our main method.
Although this is just a simple demo, in actual work, we use DelayQueue is basically this idea, but when writing code will be more general and comprehensive, let's take a look at how DelayQueue implements put and take.
3 put data
Let's take put as an example. Put calls the offer method. The source code of the offer is as follows:
public boolean offer(E e) {
final ReentrantLock lock = this.lock;
// 上锁
lock.lock();
try {
// 使用 PriorityQueue 的扩容，排序等能力
q.offer(e);
// 如果恰好刚放进去的元素正好在队列头
// 立马唤醒 take 的阻塞线程，执行 take 操作
// 如果元素需要延迟执行的话，可以使其更快的沉睡计时
if (q.peek() == e) {
leader = null;
available.signal();
}
return true;
} finally {
// 释放锁
lock.unlock();
}
}
It can be seen that the offer method of PriorityQueue is actually used at the bottom, let's take a look:
// 新增元素
public boolean offer(E e) {
// 如果是空元素的话，抛异常
if (e == null)
throw new NullPointerException();
modCount++;
int i = size;
// 队列实际大小大于容量时，进行扩容
// 扩容策略是：如果老容量小于 64，2 倍扩容，如果大于 64，50 % 扩容
if (i >= queue.length)
grow(i + 1);
size = i + 1;
// 如果队列为空，当前元素正好处于队头
if (i == 0)
queue[0] = e;
else
// 如果队列不为空，需要根据优先级进行排序
siftUp(i, e);
return true;
}
// 按照从小到大的顺序排列
private void siftUpComparable(int k, E x) {
Comparable<? super E> key = (Comparable<? super E>) x;
// k 是当前队列实际大小的位置
while (k > 0) {
// 对 k 进行减倍
int parent = (k - 1) >>> 1;
Object e = queue[parent];
// 如果 x 比 e 大，退出，把 x 放在 k 位置上
if (key.compareTo((E) e) >= 0)
break;
// x 比 e 小，继续循环，直到找到 x 比队列中元素大的位置
queue[k] = e;
k = parent;
}
queue[k] = key;
}
As you can see, the offer method of PriorityQueue mainly does three things:
- Judging the newly added elements;
- Expand the queue, the expansion strategy is very similar to the set expansion strategy;
- Sorting according to the compareTo method of elements, we hope that the final sorting results are from small to large, because we want the head of the queue to be expired data, we need to implement in the compareTo method: sort by the expiration time of each element ,as follows:
(int) (this.getDelay(TimeUnit.MILLISECONDS) - o.getDelay(TimeUnit.MILLISECONDS));
In this way, the sooner the expired elements can be ranked at the head of the team.
It can be seen that when adding data, the compareTo method is only used to sort the elements in the queue. Next, let's take a look at how to operate when fetching data.
4 Take the data
When fetching data, if you find that the expiration time of an element has expired, you can take out the data. If there is no expired element, the thread will always block. Let's take take as an example to look at the core source code:
for (;;) {
// 从队头中拿数据出来
E first = q.peek();
// 如果为空，说明队列中，没有数据，阻塞住
if (first == null)
available.await();
else {
// 获取队头数据的过期时间
long delay = first.getDelay(NANOSECONDS);
// 如果过期了，直接返回队头数据
if (delay <= 0)
return q.poll();
// 引用置为 null ，便于 gc，这样可以让线程等待时，回收 first 变量
first = null;
// leader 不为空的话，表示当前队列元素之前已经被设置过阻塞时间了
// 直接阻塞当前线程等待。
if (leader != null)
available.await();
else {
// 之前没有设置过阻塞时间，按照一定的时间进行阻塞
Thread thisThread = Thread.currentThread();
leader = thisThread;
try {
// 进行阻塞
available.awaitNanos(delay);
} finally {
if (leader == thisThread)
leader = null;
}
}
}
}
It can be seen that the blocking wait function uses the lock capability at the bottom, which we will talk about in later chapters.
The take method demonstrated above will block indefinitely, and will not return until the expiration time of the team head has expired. If you do not want to block indefinitely, you can try the poll method and set a timeout period. If the team head element has not expired within the timeout time, it will Return null.
5 Summary
DelayQueue is a very interesting queue. The bottom layer uses sorting and timeout blocking to implement a delay queue. Sorting uses PriorityQueue sorting ability. Timeout blocking uses lock waiting ability. It can be seen that DelayQueue is actually to meet the delay execution scenario. It is encapsulated on the basis of the existing API. We can learn this idea in our work and reuse the existing functions as much as possible to reduce the development workload.
