Detailed explanation of LinkedBlockingDeque, source code, principle and common methods, introduction to usage scenarios

Detailed explanation of LinkedBlockingDeque

Introduction to LinkedBlockingDeque

【1】LinkedBlockingDeque is a two-way blocking queue based on linked list. By default, the size of the blocking queue is Integer.MAX_VALUE, which can be regarded as an unbounded queue, but capacity limits can also be set as a bounded queue.

[2] Compared with other blocking queues, LinkedBlockingDeque has more methods such as addFirst, addLast, peekFirst, peekLast, etc. The method ending with first means to insert, get or remove the first element of the double-ended queue. A method ending with last means to insert, get or remove the last element of the double-ended queue. But in essence, the lock competition situation is not optimized, because whether it is from the head of the queue or the tail of the queue, they are all competing for the same lock, but there are more ways to insert and obtain data.

Source code analysis of LinkedBlockingDeque

【1】Attribute value

//典型的双端链表结构
static final class Node<E> {
    
    
    E item; //存储元素
    Node<E> prev;   //前驱节点
    Node<E> next; //后继节点

    Node(E x) {
    
    
        item = x;
    }
}
// 链表头  本身是不存储任何元素的，初始化时item指向null
transient Node<E> first;
// 链表尾
transient Node<E> last;
// 元素数量
private transient int count;
// 容量,指定容量就是有界队列
private final int capacity;
//重入锁
final ReentrantLock lock = new ReentrantLock();
// 当队列无元素时，锁会阻塞在notEmpty条件上，等待其它线程唤醒
private final Condition notEmpty = lock.newCondition();
// 当队列满了时，锁会阻塞在notFull上，等待其它线程唤醒
private final Condition notFull = lock.newCondition();

[2] Constructor

public LinkedBlockingDeque() {
    
    
    // 如果没传容量，就使用最大int值初始化其容量
    this(Integer.MAX_VALUE);
}

public LinkedBlockingDeque(int capacity) {
    
    
    if (capacity <= 0) throw new IllegalArgumentException();
    this.capacity = capacity;
}

public LinkedBlockingDeque(Collection<? extends E> c) {
    
    
    this(Integer.MAX_VALUE);
    final ReentrantLock lock = this.lock;
    lock.lock(); // 为保证可见性而加的锁
    try {
    
    
        for (E e : c) {
    
    
            if (e == null)
                throw new NullPointerException();
            //从尾部插入元素，插入失败抛出异常
            if (!linkLast(new Node<E>(e)))
                throw new IllegalStateException("Deque full");
        }
    } finally {
    
    
        lock.unlock();
    }
}

【3】Core method analysis

1) Enqueue method

//添加头结点元素
public void addFirst(E e) {
    
    
    //如果添加失败，抛出异常
    if (!offerFirst(e))
        throw new IllegalStateException("Deque full");
}

//添加尾结点元素
public void addLast(E e) {
    
    
    //如果添加失败，抛出异常
    if (!offerLast(e))
        throw new IllegalStateException("Deque full");
}

//添加头结点元素
public boolean offerFirst(E e) {
    
    
    //添加的元素为空 抛出空指针异常
    if (e == null) throw new NullPointerException();
    //将元素构造为结点
    Node<E> node = new Node<E>(e);
    //这边会加锁，并调用添加头结点插入的核心方法
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
    
    
        return linkFirst(node);
    } finally {
    
    
        //解锁
        lock.unlock();
    }
}

//添加尾结点元素
public boolean offerLast(E e) {
    
    
    //添加的元素为空 抛出空指针异常
    if (e == null) throw new NullPointerException();
    //将元素构造为结点
    Node<E> node = new Node<E>(e);
    //这边会加锁，并调用添加尾结点插入的核心方法
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
    
    
        return linkLast(node);
    } finally {
    
    
        lock.unlock();
    }
}

//头部插入
private boolean linkFirst(Node<E> node) {
    
    
    //当前容量大于队列最大容量时，直接返回插入失败
    if (count >= capacity)
        return false;
    //队列中的头结点
    Node<E> f = first;
    //原来的头结点作为 新插入结点的后一个结点
    node.next = f;
    //替换头结点 为新插入的结点
    first = node;
    //尾结点不存在，将尾结点置为当前新插入的结点
    if (last == null)
        last = node;
    else
        //原来的头结点的上一个结点为当前新插入的结点
        f.prev = node;
    //当前容量增加
    ++count;
    //唤醒读取时因队列中无元素而导致阻塞的线程
    notEmpty.signal();
    return true;
}

//尾部插入
private boolean linkLast(Node<E> node) {
    
    
    //当前容量大于队列最大容量时，直接返回插入失败
    if (count >= capacity)
        return false;
    //获取尾节点
    Node<E> l = last;
    //将新插入的前一个节点指向原来的尾节点
    node.prev = l;
    //尾结点设置为新插入的结点
    last = node;
    //头结点为空，新插入的结点作为头节点
    if (first == null)
        first = node;
    else
        //将原尾结点的下一个结点指向新插入的节点
        l.next = node;
    //当前容量增加
    ++count;
    //唤醒读取时因队列中无元素而导致阻塞的线程
    notEmpty.signal();
    return true;
}

//头结点插入
public void putFirst(E e) throws InterruptedException {
    
    
    //元素不能为空
    if (e == null) throw new NullPointerException();
    //将元素构造为结点
    Node<E> node = new Node<E>(e);
    //这边会加锁，并调用添加头结点插入的核心方法
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
    
    
        //头结点如果插入失败，会阻塞该方法，直到取出结点或删除结点时被唤醒
        while (!linkFirst(node))
            notFull.await();
    } finally {
    
    
        //解锁
        lock.unlock();
    }
}

//尾结点插入
public void putLast(E e) throws InterruptedException {
    
    
    //元素不能为空
    if (e == null) throw new NullPointerException();
    //将元素构造为结点
    Node<E> node = new Node<E>(e);
    //这边会加锁，并调用添加尾结点插入的核心方法
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
    
    
        //尾结点如果插入失败，会阻塞该方法，直到取出结点或删除结点时被唤醒
        while (!linkLast(node))
            notFull.await();
    } finally {
    
    
        lock.unlock();
    }
}

//头结点插入 可指定阻塞时间
public boolean offerFirst(E e, long timeout, TimeUnit unit)
        throws InterruptedException {
    
    
    //元素不能为空
    if (e == null) throw new NullPointerException();
    //将元素构造为结点
    Node<E> node = new Node<E>(e);
    //计算剩余应阻塞时间
    long nanos = unit.toNanos(timeout);
    //这边会加锁，并调用添加头结点插入的核心方法
    final ReentrantLock lock = this.lock;
    //获取可响应中断的锁，保证阻塞时间到期后可重新获得锁
    lock.lockInterruptibly();
    try {
    
    
        //头结点如果插入失败，会阻塞该方法，直到取出结点或删除结点时被唤醒
        //或者阻塞时间到期直接返回失败
        while (!linkFirst(node)) {
    
    
            if (nanos <= 0)
                return false;
            nanos = notFull.awaitNanos(nanos);
        }
        return true;
    } finally {
    
    
        //解锁
        lock.unlock();
    }
}

//头结点插入 可指定阻塞时间
public boolean offerLast(E e, long timeout, TimeUnit unit)
        throws InterruptedException {
    
    
    //元素不能为空
    if (e == null) throw new NullPointerException();
    //将元素构造为结点
    Node<E> node = new Node<E>(e);
    //计算剩余应阻塞时间
    long nanos = unit.toNanos(timeout);
    //这边会加锁，并调用添加尾结点插入的核心方法
    final ReentrantLock lock = this.lock;
    //获取可响应中断的锁，保证阻塞时间到期后可重新获得锁
    try {
    
    
        //尾结点如果插入失败，会阻塞该方法，直到取出结点或删除结点时被唤醒
        //或者阻塞时间到期直接返回失败
        while (!linkLast(node)) {
    
    
            if (nanos <= 0)
                return false;
            nanos = notFull.awaitNanos(nanos);
        }
        return true;
    } finally {
    
    
        //解锁
        lock.unlock();
    }
}

2) Dequeue method

//删除头结点 - 加锁 直接返回结果
public E pollFirst() {
    
    
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
    
    
        //调用删除头结点核心方法
        return unlinkFirst();
    } finally {
    
    
        lock.unlock();
    }
}

//删除尾结点 - 加锁 直接返回结果
public E pollLast() {
    
    
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
    
    
        //调用删除尾结点核心方法
        return unlinkLast();
    } finally {
    
    
        lock.unlock();
    }
}

//删除头结点 - 加锁，如果删除失败则阻塞
public E takeFirst() throws InterruptedException {
    
    
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
    
    
        E x;
        //调用删除头结点核心方法
        while ((x = unlinkFirst()) == null)
            //阻塞
            notEmpty.await();
        return x;
    } finally {
    
    
        lock.unlock();
    }
}

//删除尾结点 - 加锁，如果删除失败则阻塞
public E takeLast() throws InterruptedException {
    
    
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
    
    
        E x;
        //调用删除尾结点核心方法
        while ((x = unlinkLast()) == null)
            //阻塞
            notEmpty.await();
        return x;
    } finally {
    
    
        lock.unlock();
    }
}

//删除头结点 - 加锁，如果删除失败则阻塞 可以指定阻塞时间
public E pollFirst(long timeout, TimeUnit unit)
        throws InterruptedException {
    
    
    //计算应阻塞时间
    long nanos = unit.toNanos(timeout);
    final ReentrantLock lock = this.lock;
    lock.lockInterruptibly();
    try {
    
    
        E x;
        //调用删除头结点核心方法
        while ((x = unlinkFirst()) == null) {
    
    
            if (nanos <= 0)
                //阻塞时间过期，返回结果
                return null;
            //阻塞 并指定阻塞时间
            nanos = notEmpty.awaitNanos(nanos);
        }
        return x;
    } finally {
    
    
        lock.unlock();
    }
}

//删除尾结点 - 加锁，如果删除失败则阻塞 可以指定阻塞时间
public E pollLast(long timeout, TimeUnit unit)
        throws InterruptedException {
    
    
    //计算应阻塞时间
    long nanos = unit.toNanos(timeout);
    final ReentrantLock lock = this.lock;
    lock.lockInterruptibly();
    try {
    
    
        E x;
        //调用删除尾结点核心方法
        while ((x = unlinkLast()) == null) {
    
    
            if (nanos <= 0)
                //阻塞时间过期，返回结果
                return null;
            //阻塞 并指定阻塞时间
            nanos = notEmpty.awaitNanos(nanos);
        }
        return x;
    } finally {
    
    
        lock.unlock();
    }
}

//删除头结点
private E unlinkFirst() {
    
    
    //获取当前头结点
    Node<E> f = first;
    //头结点为空 返回空
    if (f == null)
        return null;
    //获取头结点的下一个结点
    Node<E> n = f.next;
    //获取头结点元素(记录return需要用到的删除了哪个元素)
    E item = f.item;
    //将头结点元素置为null
    f.item = null;
    //将头结点的下一个结点指向自己 方便gc
    f.next = f;
    //设置头结点为原头结点的下一个结点
    first = n;
    //若原头结点的下一个结点不存在（队列中没有了结点）
    if (n == null)
        //将尾结点也置为null
        last = null;
    else
        //新的头结点的前一个结点指向null，因为他已经作为了头结点 所以不需要指向上一个结点
        n.prev = null;
    //当前数量减少
    --count;
    //唤醒因添加元素时队列容量满导致阻塞的线程
    notFull.signal();
    //返回原来的头结点中的元素
    return item;
}

//删除尾结点
private E unlinkLast() {
    
    
    //获取当前尾结点
    Node<E> l = last;
    //尾结点不存在 返回null
    if (l == null)
        return null;
    //获取当前尾结点的上一个结点
    Node<E> p = l.prev;
    //获取当前尾结点中的元素，需要返回记录
    E item = l.item;
    //将当前尾结点的元素置为null
    l.item = null;
    //并将当前尾结点的上一个结点指向自己，方便gc
    l.prev = l;
    //设置新的尾结点，为原来尾结点的前一个结点
    last = p;
    //若无新的尾结点，头结点置为空（队列中没有了结点）
    if (p == null)
        first = null;
    else
        //将新的尾结点的下一个结点指向null，因为他已经为尾结点所以不需要指向下一个结点
        p.next = null;
    //数量减少
    --count;
    //唤醒因添加元素时队列容量满导致阻塞的线程
    notFull.signal();
    //返回原来的尾结点元素
    return item;
}

LinkedBlockingDequeSummary

[1] A linked list blocks the double-ended unbounded queue, and the capacity can be specified, and the default is Integer.MAX_VALUE

[2] Data structure: linked list (same as LinkedBlockingQueue, internal class Node storage elements)

[3] Lock: ReentrantLock (same as ArrayBlockingQueue) access is the same lock, and the operation is the same array object

[4] Blocking object (notEmpty [exit: queue count=0, when no element is available, block on this object], notFull [enqueue: queue count=capacity, when no element can be put in, block on this object] )

【5】Join the team, both the beginning and the end can be added and deleted.

【6】Out of the team, both the beginning and the end can be added and deleted.

【7】Application scenario: Commonly used in "work stealing algorithm".

---

---

A simple application of LinkedBlockingDeque

Background: In a certain product code, when an event is generated, an event id will be generated, and the product will use the event id for subsequent logical processing.

Requirement: Now there is another requirement, to get the event id of each event, and then process other logic in another class. Because the original logic processing is more, the new requirements can only minimize the embedding of the original code.

Idea: Create a class to handle new requirements. This class provides a method A() to store the event id in the queue. In the code of the original product, you only need to call the A() method to store the event id in the queue, and then you can use it in the new Logical processing in the class.

A brief introduction to LinkedBlockingDeque (this article is just a simple record of one use)

• LinkedBlockingDeque is a two-way blocking queue composed of a linked list structure, that is, elements can be inserted and removed from both ends of the queue.
• Compared with other blocking queues, LinkedBlockingDeque has more methods such as addFirst, addLast, peekFirst, peekLast, etc.
• LinkedBlockingDeque has an optional capacity, and the default capacity is Integer.MAX_VALUE.

Use process

1. First create a CommenServiceImpl class as a class to handle new requirements

@Service("GtmcCommenServiceImpl")
public class CommenServiceImpl{

//omit

}

2. Create a LinkedBlockingDeque object incidentIdList, and add a new method to add event id to incidentIdList

final static BlockingDeque<String> incidentIdList = new LinkedBlockingDeque<>();
private volatile Thread thread = null;
//当有事件id添加成功后，起一个线程单独处理新的逻辑
public void SetIncidentIdList(String incidentId) {
    if (null == incidentId || "".equals(incidentId.trim())) {
        return;
    }
    incidentIdList.offer(incidentId);
    log.info("事件id入List成功：" + incidentId);
    if (thread == null) {
        thread = new Thread(this::setAlarm);
        thread.start();
    }
    if(!thread.isAlive() || thread.isInterrupted()){
        thread = new Thread(this::setAlarm);
        thread.start();
    }
}

3. Inject CommenServiceImpl into the product code that originally generated the event id , and then add the event id to LinkedBlockingDeque through the SetIncidentIdList method

gtmcCommenService.SetIncidentIdList(incidentId);

4. Process the new requirement logic in the newly created CommenServiceImpl class

@Slf4j
@Service("CommenServiceImpl")
public class CommenServiceImpl {
    
    
    protected static ElasticsearchService elasticsearchService;
    @org.springframework.beans.factory.annotation.Value("${douc.ex.accountId}")
    private String accountId;
    @Autowired
    private IncidentService incidentService;
    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;
    @Autowired
    protected EsQueryCreator queryCreator;
    final static BlockingDeque<String> incidentIdList = new LinkedBlockingDeque<>();
  
    private volatile Thread thread = null;
 
    public GtmcCommenServiceImpl() {
    
    
        elasticsearchService = ElasticsearchService.getInstance();
    }
 
    /**
     * 新起一个线程循环处理
     */
    void setAlarm() {
    
    
        while (true) {
    
    
            try {
    
    
                List<String> list = new LinkedList<>();
                incidentIdList.drainTo(list,20);//一次最多从incidentIdList拿20个出来
                for (String evenId : list) {
    
    
                    
 
                   //省略逻辑处理代码
 
 
 
                    kafkaTemplate.send(TOPIC, JSONObject.toJSONString(map));
                    
                }
            } catch (Exception e) {
    
    
                log.error("监控告警生成和解决，发送事件id和配置项id给kafka发生错误 {}" + (e));
            }
        }
    }
 
    public void SetIncidentIdList(String incidentId) {
    
    
        if (null == incidentId || "".equals(incidentId.trim())) {
    
    
            return;
        }
        incidentIdList.offer(incidentId);
        log.info("事件id入List成功：" + incidentId);
        if (thread == null) {
    
    
            thread = new Thread(this::setAlarm);
            thread.start();
        }
        if(!thread.isAlive() || thread.isInterrupted()){
    
    
            thread = new Thread(this::setAlarm);
            thread.start();
        }
    }

---

---

LinkedBlockingDeque source code, principles and common methods, usage scenario introduction

Overview and Introduction

LinkedBlockingDeque is a bidirectional optional bounded blocking queue based on linked list provided by juc package .

Creating bounded queues by passing arguments to constructors prevents excessive bloat. If the capacity is not specified, the default maximum capacity is Integer#MAX_VALUE =2^31-1.

Each node of the linked list is dynamically created when inserting.

If there is no blocking, most operations can be completed in constant time. Except for #remove(Object), removeFirstOccurrence, removeLastOccurrence, contains, iterator methods and batch operations, the time complexity of these operations is O(n).

The characteristic of blocking queue is that if the queue is empty, take/pop an element will wait forever (remove will report an error, pull will return null, poll depends on the situation, get will throw an exception); if the queue is full, offer/put an element will also It will block and wait forever (add will falsely report that the capacity is full).

data structure

The data structure itself is not very complicated. First, there is an inner class final class Node, which is used to wrap the data of each node. In addition, there are member attributes Node first, Node last, the number of elements currently owned int count, and the capacity int capacity;

This is common, as well as the ReentrantLock lock used to implement blocking and the corresponding two Conditions, notEmpty and notFull .

Node

A final inner class with only three member variables

E item 当前节点数据
Node<E> prev  指向当前节点前驱节点的地址
Node<E> next  指向当前节点下个节点的地址

common method

No parameter constructor

The default queue length of the object returned by the no-argument constructor is the maximum value of Integer. 2^32-1

Construction method LinkedBlockingDeque(Collection<? extends E> c)

   public LinkedBlockingDeque(Collection<? extends E> c) {
    
    
        this(Integer.MAX_VALUE);
        final ReentrantLock lock = this.lock;
        lock.lock(); // Never contended, but necessary for visibility
        try {
    
    
            for (E e : c) {
    
    
                if (e == null)
                    throw new NullPointerException();
                if (!linkLast(new Node<E>(e)))
                    throw new IllegalStateException("Deque full");
            }
        } finally {
    
    
            lock.unlock();
        }
    }

Capacity is the maximum value of Integer

lock.lock() before operation

try {} finally {} to release the lock,

Take each element and place it at the end of the linked list in turn

Add an element void addFirst(E e)

        if (e == null) throw new NullPointerException();
        Node<E> node = new Node<E>(e);
        final ReentrantLock lock = this.lock;
        lock.lock();
        try {
    
    
            return linkFirst(node);
        } finally {
    
    
            lock.unlock();
        }

The key linkFirst method

    private boolean linkFirst(Node<E> node) {
    
    
        // assert lock.isHeldByCurrentThread();
        if (count >= capacity)
            return false;
        Node<E> f = first;
        node.next = f;
        first = node;
        if (last == null)
            last = node;
        else
            f.prev = node;
        ++count;
        notEmpty.signal();
        return true;
    }

The main logic is to first judge whether the current capacity is exceeded, then point the next of the current node to the original first, and point the prev of the original first to the current node, and note that if the first element comes in, initialize the last pointer.

After the conventional linked list is operated, it is necessary to reflect the characteristics of the blocking queue. If a thread is performing an operation similar to take, it may be blocked and waiting, so it is necessary to use Condition notEmpty to notify it so that it can be woken up and perform corresponding operations.

boolean offerFirst(E e)/boolean offerLast(E e)

The difference between this and the add method is that if the capacity is full but not added, no error will be reported directly, and false will only be returned

putFirst(E e)/putLast(E e)

These two methods will block when the queue is full, which is where the above methods differ.

    public void putFirst(E e) throws InterruptedException {
    
    
        if (e == null) throw new NullPointerException();
        Node<E> node = new Node<E>(e);
        final ReentrantLock lock = this.lock;
        lock.lock();
        try {
    
    
            while (!linkFirst(node))
                notFull.await();
        } finally {
    
    
            lock.unlock();
        }
    }

A key difference is that if the linkFirst attempt fails, it will await and wait for an element signal (meaning notify) to be removed.

Another point is that if multiple threads are awaiting here, taking out an element will only signal a certain thread.

boolean offerFirst/offerLast(E e, long timeout, TimeUnit unit)

Inserting an element at the front or the end of the queue will wait for the specified time, and if it fails after the specified time, it will return false.

Take out an element E removeFirst()/removeLast()

    public E removeFirst() {
    
    
        E x = pollFirst();
        if (x == null) throw new NoSuchElementException();
        return x;
    }

Key method E unlinkFirst()

    private E unlinkFirst() {
    
    
        // assert lock.isHeldByCurrentThread();
        Node<E> f = first;
        if (f == null)
            return null;
        Node<E> n = f.next;
        E item = f.item;
        f.item = null;
        f.next = f; // help GC
        first = n;
        if (n == null)
            last = null;
        else
            n.prev = null;
        --count;
        notFull.signal();
        return item;
    }

Take out the first one, and then mark that the queue is definitely not full, notFull.signal(); If you want to put/offer, you can continue.

E pollFirst()/pollLast()

It is similar to the remove method, except that it will not throw a null pointer and return null directly

E pollFirst(long timeout, TimeUnit unit)/pollLast(long timeout, TimeUnit unit)

The design of this overloaded method is very strange. If a timeout is given, it will wait for the specified time instead, and return null if it has not been retrieved by the time.

E takeFirst()/takeLast()

    public E takeLast() throws InterruptedException {
    
    
        final ReentrantLock lock = this.lock;
        lock.lock();
        try {
    
    
            E x;
            while ( (x = unlinkLast()) == null)
                notEmpty.await();
            return x;
        } finally {
    
    
            lock.unlock();
        }
    }

If you don't get it, you will wait forever.

E getFirst()/getLast()

This is nothing, if the fetch is null, an exception will be thrown

E peekFirst()/peekLast()

This does not change the queue itself, and knowledge checks the beginning and end of the operation

    public E peekFirst() {
    
    
        final ReentrantLock lock = this.lock;
        lock.lock();
        try {
    
    
            return (first == null) ? null : first.item;
        } finally {
    
    
            lock.unlock();
        }
    }

The other methods will not be expanded one by one, they are relatively common.

Application Scenarios and Extension Points

JDK provides 7 blocking queues

• ArrayBlockingQueue : A bounded blocking queue composed of array structures.
• LinkedBlockingQueue: A bounded blocking queue composed of a linked list structure.
• PriorityBlockingQueue : An unbounded blocking queue that supports priority sorting.
• DelayQueue: An unbounded blocking queue implemented using a priority queue.
• SynchronousQueue: A blocking queue that does not store elements.
• LinkedTransferQueue: An unbounded blocking queue composed of a linked list structure.
• LinkedBlockingDeque: A bidirectional blocking queue composed of a linked list structure.

Blocking queue is a typical producer-consumer pattern, which is very similar to middleware MQ, consuming while producing. Doing so can decouple producers and consumers.

When in use, the object wrapped by Node is basically an object that implements Runnable. For example, when creating a thread pool, there is a parameter that is a blocking queue, which is used to save the submitted tasks, and then the worker threads in the thread pool will not Stop to get tasks from the blocking queue for processing.