ConcurrentLinkedQueue of concurrent containers

1. Introduction to ConcurrentLinkedQueue

In single-threaded programming, we often use some collection classes, such as ArrayList, HashMap, etc., but these classes are not thread-safe classes. There are often some test points in the interview, such as ArrayList is not thread-safe, Vector is thread-safe. The way to ensure the thread safety of Vector is to use a synchronized exclusive lock on the method to turn multi-threaded execution into serialization. To make ArrayList thread-safe, you can also use the Collections.synchronizedList(List<T> list)method ArrayList to convert to thread-safe, but this conversion method is still implemented through the synchronized modification method. Obviously, this is not an efficient method. At the same time, queues are also commonly used by us. In order to solve the problem of thread safety, Master Doug Lea prepared ConcurrentLinkedQueue, a thread-safe queue for us. It can be seen from the class name that the data structure implementing the queue is chained.

1.1 Node

If you want to learn ConcurrentLinkedQueue first, you must first look at its node class and understand its underlying data structure. The source code of the Node class is:

private static class Node<E> {
        volatile E item;
        volatile Node<E> next;
		.......
}

The Node node mainly contains two fields: one is the data field item, and the other is the next pointer, which is used to point to the next node to form a chained queue. And they are all decorated with volatile to ensure memory visibility ( see this article about volatile ). In addition, ConcurrentLinkedQueue contains such two member variables:

private transient volatile Node<E> head;
private transient volatile Node<E> tail;

Explain that ConcurrentLinkedQueue manages the queue by holding head and tail pointers. When we call the no-argument constructor, its source code is:

public ConcurrentLinkedQueue() {
    head = tail = new Node<E>(null);
}

The head and tail pointers will point to a node whose item field is null. At this time, the ConcurrentLinkedQueue state is shown in the following figure:

As shown in the figure, head and tail point to the same node Node0, the item field of this node is null, and the next field is null.

1.2 Several CAS operations that operate Node

When the queue is dequeued and enqueued, it is inevitable that the node needs to be operated, and the problem of thread safety is easy to occur in multi-threading. It can be seen that after the processor instruction set can support the CMPXCHG instruction, the CAS operation will be used in the concurrent processing in the java source code (for the CAS operation, please refer to Section 3.1 of this article ), then the CAS operation of the Node in the ConcurrentLinkedQueue has A few like this:

//更改Node中的数据域item	
boolean casItem(E cmp, E val) {
    return UNSAFE.compareAndSwapObject(this, itemOffset, cmp, val);
}
//更改Node中的指针域next
void lazySetNext(Node<E> val) {
    UNSAFE.putOrderedObject(this, nextOffset, val);
}
//更改Node中的指针域next
boolean casNext(Node<E> cmp, Node<E> val) {
    return UNSAFE.compareAndSwapObject(this, nextOffset, cmp, val);
}

It can be seen that these methods are actually methods by calling the UNSAFE instance. UNSAFE is the sun.misc.Unsafe class, which is the underlying method of hotspot. So far, it is enough to know that the operation of CAS is provided by this class in the final analysis.

2. offer method

For a queue, insertion satisfies the FIFO characteristic, inserting elements are always inserted at the end of the queue, and fetching (removing) elements is always from the head of the queue. All to be able to fully understand ConcurrentLinkedQueue naturally starts from the offer method and the poll method. Then in order to understand the offer method, we use the debug method to see the code line by line. In addition, when looking at multithreaded code, you can use this way of thinking:

A single thread offers multiple threads offer some thread offers, and some threads poll ---- the speed of the offer is faster than the poll --------- The queue length will become longer and longer, because the offer node is always at the end of the queue. , and the poll node is always at the head of the queue, that is to say, there is no "intersection" between the offer thread and the poll thread, that is to say, the two types of threads do not affect each other. This situation is from the perspective of relative speed. , that is, a "single-threaded offer" -- the speed of the offer is slower than that of the poll -- the relative rate of the poll is faster than the offer, that is, the speed of deleting the head of the team is faster than adding nodes at the tail of the team The result is that the queue length will become shorter and shorter, and the offer thread and the poll thread will have an "intersection", that is, at that moment, the node that the offer thread and the poll thread operate at the same time can be called the critical point , and At this node, the offer thread and the poll thread must affect each other. According to the relative order of offer and poll at the critical point, we can think from two perspectives: 1. The execution order is offer-->poll-->offer , that is, when the offer thread inserts Node2 after Node1, this When the poll thread has deleted Node1, this situation obviously needs to be considered in the offer method; 2. The execution order may be: poll-->offer-->poll , that is, when the node to be deleted by the poll thread is null (The queue is an empty queue), at this time the offer thread inserts a node to make the queue a non-empty queue

First look at this piece of code:

1. ConcurrentLinkedQueue<Integer> queue = new ConcurrentLinkedQueue<>();
2. queue.offer(1);
3. queue.offer(2);

Create a ConcurrentLinkedQueue instance, offer 1 first, then offer 2. The source code of offer is:

public boolean offer(E e) {
1.    checkNotNull(e);
2.    final Node<E> newNode = new Node<E>(e);

3.    for (Node<E> t = tail, p = t;;) {
4.        Node<E> q = p.next;
5.        if (q == null) {
6.            // p is last node
7.            if (p.casNext(null, newNode)) {
                // Successful CAS is the linearization point
                // for e to become an element of this queue,
               // and for newNode to become "live".
8.                if (p != t) // hop two nodes at a time
9.                    casTail(t, newNode);  // Failure is OK.
10.                return true;
            }
            // Lost CAS race to another thread; re-read next
        }
11.        else if (p == q)
            // We have fallen off list.  If tail is unchanged, it
            // will also be off-list, in which case we need to
            // jump to head, from which all live nodes are always
            // reachable.  Else the new tail is a better bet.
12.            p = (t != (t = tail)) ? t : head;
           else
            // Check for tail updates after two hops.
13.            p = (p != t && t != (t = tail)) ? t : q;
    }
}

Single-threaded execution perspective analysis :

From the perspective of single- threaded execution , analyze the process of offer 1. The first line of code will judge whether it is null. If it is null, a null pointer exception will be thrown directly. The second line of code will wrap e into a Node class, and the third line of code will be a for loop. There are only initialization conditions and no loop end conditions. It is in line with the "routine" of CAS. If the CAS operation in the loop body is successful, it will return directly. If the CAS operation fails, it will continue to retry in the for loop until it succeeds. Here the instance variable t is initialized to tail and p is initialized to t which is tail. In order to facilitate the following understanding, p is regarded as the real tail node of the queue, and tail does not necessarily point to the real tail node of the object, because tail is delayed in the ConcurrentLinkedQueue update, and we will take a look at the specific reasons. When the code reaches line 3, both t and p point to Node0 whose item field is null and next field is null, respectively, created during initialization. The variable q in line 4 is assigned to null, the if in line 5 is judged to be true, and in line 7, casNext is used to set the inserted Node as the next node of the current queue tail node p. If the CAS operation fails, the loop ends at the next time. Retry in a loop. The CAS operation successfully goes to the 8th line. At this time, p==t, if the judgment is false, and the return true is returned directly. If 1 is successfully inserted, the status of the ConcurrentLinkedQueue at this time is shown in the following figure:

As shown in the figure, the tail node of the queue should be Node1 at this time, and the node pointed to by tail is still Node0, so it can be shown that tail is updated with a delay. Then we continue to look at the situation of offer 2. Obviously, the node pointed to by q in line 4 is not null at this time, but points to Node1, the if in line 5 is judged as false, and the if in line 11 is judged as false, the code will go to line 13. Well, when we insert the node again, we will ask ourselves such a question? It has been explained above that tail does not point to the real tail node of the queue, so when inserting a node, should we first do is to find where the current tail node of the queue can be inserted? Then the 13th line of code is to find the real tail node of the queue .

Locate the real opposite tail node of the queue

p = (p != t && t != (t = tail)) ? t : q;

Let's analyze this line of code. If this code is executed in a single- threaded environment , it is obvious that since p==t, p will be assigned to q at this time, and q is equal to Node<E> q = p.nextNode1. In the first loop, the pointer p points to the real queue node Node1, then in the next loop, the node pointed to by the fourth line q is null, then in the fifth line if the judgment is true, then the seventh line is still Set the next of the p node to the currently added Node through the casNext method, and then go to the 8th line. At this time, p!=t, the 8th line if is judged to be true, and casTail(t, newNode)the current node Node will be set to the end of the queue by setting the queue. Node, the schematic diagram of the queue status at this time is shown in the following figure:

The node pointed to by tail is changed from Node0 to Node2 . The reason why casTail fails and does not need to be retried is that the next Node<E> q = p.nextlogical direction of the offer code is mainly determined by the next node q( ) of p. When casTail fails, the state diagram is as follows:

As shown in the figure, if casTail fails to set tail, that is, tail still points to Node0 node, it is nothing more than looping several times to locate the tail node through 13 lines of code .

By analyzing the single-threaded execution angle, we can understand that the execution logic of poll is:

1. If the next node (next field) of the node pointed to by tail is null, it means that the node pointed to by tail is the real tail node of the queue, so the node to be inserted can be inserted through casNext, but tail does not change at this time. As shown in Figure 2;

2. If the next node (next field) of the node pointed to by tail is not null, it means that the node pointed to by tail is not the real tail node of the queue. Pass the q（Node<E> q = p.next）pointer forward to find the node at the end of the queue, then insert the node to be inserted through casNext, and change the tail through casTail, as shown in Figure 3 .

Let's go back and see p = (p != t && t != (t = tail)) ? t : q;that this line of code is in a single thread. This code will never assign p to t, so writing it like this will have no effect. Then we try to analyze it in a multi-threaded situation.

Analysis from the perspective of multi-threaded execution

Multiple thread offers

Obviously, there is another deep meaning in this writing. In fact , this line of code is very interesting in a multi-threaded environment . t != (t = tail)This operation is not an atomic operation , there is such a case:

As shown in the figure, assuming that thread A reads the variable t at this time, and thread B just offers a Node at this time, it will modify the tail pointer at this time, then at this time, when thread A executes t=tail again, t will point to another node, Obviously, the nodes pointed to by the variable t read twice before and after thread A are different, t != (t = tail)that true, and because the change of the node pointed to by t p != tis also true, the execution result of this line of code at this time is that the latest t pointer of p and t points to the same node, and at this time t is also the real tail node of the queue. Then, now that the real tail node of the queue has been located, the offer operation can be performed.

offer->poll->offer

Then we still have no analysis of the code on line 11. It can be roughly guessed that it should answer some thread offers and some polls . When it if (p == q)is true, it means that the next node of the node pointed to by p also points to itself. This kind of node is called a sentinel node . This kind of node has little value in the queue, and is generally expressed as a node to be deleted or an empty node . In order to understand this situation well, let's first look at the execution process of the poll method, and then look back. In short, this is a very interesting thing :).

3. poll method

The source code of the poll method is as follows:

public E poll() {
    restartFromHead:
    1. for (;;) {
    2.    for (Node<E> h = head, p = h, q;;) {
    3.        E item = p.item;

    4.        if (item != null && p.casItem(item, null)) {
                // Successful CAS is the linearization point
                // for item to be removed from this queue.
    5.            if (p != h) // hop two nodes at a time
    6.                updateHead(h, ((q = p.next) != null) ? q : p);
    7.            return item;
            }
    8.        else if ((q = p.next) == null) {
    9.            updateHead(h, p);
    10.            return null;
            }
    11.        else if (p == q)
    12.            continue restartFromHead;
            else
    13.            p = q;
        }
    }
}

Let's first clarify the basic logic of this method from a single-threaded perspective . Suppose the initial state of ConcurrentLinkedQueue is as shown in the following figure:

For the definition of the parameter offer, we still use the variable p as the queue to delete the real head node of the queue. The node pointed to by h (head) is not necessarily the head node of the queue . Let's first look at the situation when polling Node1. Since p=h=head, referring to the above figure, it is obvious that the data field of Node1 pointed to by p is not null at this time. After the 4th line of code is item!=nulljudged to be true, the next step is casItemto set the data field of Node1. is null. If the CAS setting fails, this cycle ends and waits for the next cycle to retry. If the execution of line 4 is successful and it enters the code of line 5, at this time both p and h point to Node1, the if in line 5 is judged to be false, and then go directly to line 7 to return to the data field 1 of Node1, and the method ends, at this time The queue status is as shown below.

Continue to poll from the queue. Obviously, the current data field of Node1 pointed to by h and p is null, so the first thing is to locate the queue head node to be deleted (find the node whose data field is not null) .

Locate the deleted head node

Continue to look, the third line of code item is null, the fourth line of code if is judged to be false, and the 8th line of code ( q = p.next) if is also false, since q points to Node2, the if judgment on the 11th line is also false, Therefore, the code goes to line 13. At this time, p and q point to Node2 together, and the real head node to be deleted is found. It can be concluded that the process of locating the queue head node to be deleted is as follows: if the data field of the current node is null, it is obvious that the node is not the node to be deleted, and the next node of the current node is used to test . After the first cycle, the state diagram is as follows:

Carry out the next cycle, the operation of line 4 is the same as the above, currently assuming that the casItem setting in line 4 is successful, since p has pointed to Node2, and h still points to Node1, at this time, the if in line 5 is judged to be true, and then execute updateHead(h, ((q = p.next) != null) ? q : p), at this time, the Node3 pointed to by q, all passed to the updateHead method are the h reference pointing to Node1 and the q reference pointing to Node3. The source code of the updateHead method is:

final void updateHead(Node<E> h, Node<E> p) {
    if (h != p && casHead(h, p))
        h.lazySetNext(h);
}

This method works primarily by casHeadpointing the head of the queue to Node3, and h.lazySetNextby pointing the next field of Node1 to itself. Finally return the value of Node2 in line 7 of code. The status of the queue at this time is shown in the following figure:

Node1's next field points to itself, and head points to Node3. If the queue is an empty queue, it will be executed to the 8th line of the code , and the (q = p.next) == nullif is judged to be true, so null is returned directly in the 10th line. The above analysis is from the perspective of single-threaded execution, and it also allows us to understand the overall idea of ​​poll. Now let's make a summary:

1. If the Item of the node pointed to by head, h and p is not null, it means that the node is the real head node (node ​​to be deleted), just set the item field to null through the casItem method, and then set the original item Just go back.

2. If the item of the node pointed to by head, h and p is null, it means that the node is not the real node to be deleted, so what should be done is to find the node whose item is not null. Test by letting q point to the next node of p (q = p.next), and if found, update the node pointed to by head and construct a sentinel node ( 通过updateHead方法的h.lazySetNext(h)) through the updateHead method .

Next, according to the way of thinking of analyzing the offer above, let's analyze the situation of multi-threading. The first situation is;

Analysis of multi-threaded execution:

Multiple thread poll

Now looking back at the source code of the poll method, there is this part:

else if (p == q)
    continue restartFromHead;

This part is to deal with the case of multiple thread polls, q = p.nextthat is to say, q always points to the next node of p, then under what circumstances will p and q point to the same node? According to our analysis above, only the node pointed to by p is turned into a sentinel node (via h.lazySetNext in the updateHead method) when polling. When thread A is judging p==q, thread B has already executed the poll method to convert the node pointed to by p to a sentinel node and the node pointed to by head has changed, so it needs to be executed from restartFromHead to ensure that the latest head is used. .

poll->offer->poll

Imagine, there is such a situation, if the current queue is empty, thread A performs the poll operation, while thread B performs the offer, and then thread A is performing poll, then at this time, thread A returns null or thread B just inserted it. What about the newest node? Let's write a generation demo:

public static void main(String[] args) {
    Thread thread1 = new Thread(() -> {
        Integer value = queue.poll();
        System.out.println(Thread.currentThread().getName() + " poll 的值为：" + value);
        System.out.println("queue当前是否为空队列：" + queue.isEmpty());
    });
    thread1.start();
    Thread thread2 = new Thread(() -> {
        queue.offer(1);
    });
    thread2.start();
}

The output is:

The value of Thread-0 poll is: null Whether the queue is currently empty queue: false

Control the execution order of thread thread1 and thread thread2 through debug. Thread1 first executes to the 8th line of code if ((q = p.next) == null). Since the queue is empty at this time, if the queue is judged to be true, it enters the if block. At this time, thread1 is suspended first, and then thread2 inserts the offer value. After the node is 1, thread2 execution ends. Let thread1 execute again. At this time, thread1 does not retry , but the code continues to go down and returns null, although at this time the queue has inserted a new node with a value of 1 due to thread2. So the output result of thread0 poll is null, but the queue is not an empty queue. Therefore, when judging whether the queue is an empty queue, it cannot be judged by the thread returning null when polling, but it can be judged by the isEmpty method .

4. Some threads offer some threads poll in the offer method

When analyzing the offer method, we also left a question, that is, the understanding of the 11th line of code in the offer method.

offer->poll->offer

In the 11th line of the offer method, the node pointed to by p can be a sentinel nodeif (p == q) when the if is judged to be true , and when will the sentinel node be constructed? In the discussion of the poll method, we have found the answer, that is, when the item field of the node pointed to by head is null, the real head node will be searched, and after the node to be inserted is inserted, the head will be updated, and the The node pointed to by the original head is set as the sentinel node. **Assume the initial state of the queue as shown in the following figure:

Therefore, when thread A executes the offer, thread B executes the poll, and there will be one of the following situations:

As shown in the figure, the tail node of thread A has the next node Node1, so it will look for the real tail node of the queue by referring to q. When the judgment is executed if (p == q), thread B performs the poll operation. For thread B, head And p points to Node0. Since the item field of Node0 is null, it will also advance forward to find the real head node Node1 of the queue. After thread B executes poll, Node0 will be converted to a sentinel node , which means that the queue's The head has changed, and the queue status is as shown below.

At this time, thread A if (p == q)is true when executing the judgment, and will continue to execute p = (t != (t = tail)) ? t : head;. Since the tail pointer has not changed, p is assigned to head, and the insertion operation is completed from the head again.

5. Design of HOPS

Through the analysis of the offer and poll methods above, we find that the tail and head are updated with a delay, and the trigger timing for the two updates is:

Tail update trigger timing : When the next node of the node pointed to by tail is not null, the operation of locating the real tail node of the queue will be performed. After the tail node is found, the tail will be updated through casTail after the insertion is completed; when tail When the next node of the pointed node is null, only the node is inserted and the tail is not updated.

**Head update trigger timing: **When the item field of the node pointed to by head is null, the operation of locating the real head node of the queue will be performed. After the head node is found and the deletion is completed, the head update will be performed through updateHead; When the item field of the node pointed to by head is not null, only the node is deleted without updating the head.

And during the update operation, there will be a comment in the source code: hop two nodes at a time . So the reason why this delayed update strategy is called HOPS is this (guess:)). As can be seen from the state diagram of the above update, the update of head and tail is "jumping", that is, there is always an interval in the middle got one. So what is the intent of this design?

If tail is always used as the tail node of the queue, the amount of code to be implemented will be less, and the logic will be easier to understand. However, this has a disadvantage. **If a large number of enqueuing operations are performed, CAS is performed every time to update the tail, which will also result in a great loss of performance. If the operation of CAS update can be reduced, the operation efficiency of joining the queue can be greatly improved, so master doug lea only uses CAS to update the tail every interval (the distance between the tail and the tail node is 1). **The same is true for the update of the head. Although this design will locate the tail node in the loop, the overall read operation efficiency is much higher than the write performance. Therefore, the extra is in the loop. The performance penalty for locating the tail node in the middle is relatively small.

References

The Art of Java Concurrent Programming

"Java High Concurrency Programming"

ConcurrentLinkedQueue Hirofumi: https://www.cnblogs.com/sunshine-2015/p/6067709.html