Today we are going to discuss another class of containers: blocking queues.
The queues we have contacted in the front are all non-blocking queues, such as PriorityQueue and LinkedList (LinkedList is a doubly linked list, which implements the Dequeue interface).
A big problem when using a non-blocking queue is that it will not block the current thread, so in the face of a consumer-producer model, you must additionally implement a synchronization strategy and an inter-thread wake-up strategy. This It is very troublesome to implement. But with a blocking queue, it is different. It will block the current thread. For example, if a thread takes an element from an empty blocking queue, the thread will be blocked until there is an element in the blocking queue. When there is an element in the queue, the blocked thread will be woken up automatically (no need for us to write code to wake up). This provides great convenience.
This article first describes the main types of blocking queues provided under the java.util.concurrent package, then analyzes the methods of blocking queues and non-blocking queues, then analyzes the implementation principle of blocking queues, and finally gives a practical example and several usage scenarios.
1. Several main blocking queues
2. Methods in blocking queues VS methods in non-blocking queues
3. Implementation principle of blocking queue
4. Examples and usage scenarios
1. Several main blocking queues
Since Java 1.5, several blocking queues have been provided under the java.util.concurrent package, mainly as follows:
ArrayBlockingQueue: A blocking queue implemented based on an array, the capacity size must be specified when creating an ArrayBlockingQueue object. And you can specify fairness and unfairness. By default, it is unfair, that is, it is not guaranteed that the thread with the longest waiting time can access the queue first.
LinkedBlockingQueue: A blocking queue implemented based on a linked list. If the capacity size is not specified when the LinkedBlockingQueue object is created, the default size is Integer.MAX_VALUE.
PriorityBlockingQueue: The above two queues are first-in-first-out queues, but PriorityBlockingQueue is not. It sorts the elements according to the priority of the elements, and dequeues according to the priority order. The elements that are dequeued each time are the elements with the highest priority. . Note that this blocking queue is an unbounded blocking queue, that is, the capacity has no upper limit (you can know from the source code that it does not have a signal sign that the container is full), and the first two are bounded queues.
DelayQueue: Based on PriorityQueue, a delay blocking queue, the elements in the DelayQueue can only be obtained from the queue when the specified delay time is up. DelayQueue is also an unbounded queue, so the operation (producer) that inserts data into the queue will never be blocked, and only the operation (consumer) that gets data will be blocked.
2. Methods in blocking queues VS methods in non-blocking queues
1. Several main methods in non-blocking queues:
add(E e): insert element e at the end of the queue, if the insertion is successful, it will return true; if the insertion fails (that is, the queue is full), an exception will be thrown;
remove(): remove the first element of the queue, if the removal is successful, it will return true; if the removal fails (the queue is empty), an exception will be thrown;
offer(E e): insert element e at the end of the queue, if the insertion is successful, it returns true; if the insertion fails (that is, the queue is full), it returns false;
poll(): remove and get the first element of the team, if successful, return the first element of the team; otherwise, return null;
peek(): Get the first element of the team, if successful, return the first element of the team; otherwise, return null
For non-blocking queues, it is generally recommended to use offer, poll and peek methods, and add and remove methods are not recommended. Because the three methods of offer, poll and peek can use the return value to determine whether the operation is successful or not, but the add and remove methods cannot achieve such an effect. Note that none of the methods in the non-blocking queue are synchronized.
2. Several main methods in the blocking queue:
The blocking queue includes most of the methods in the non-blocking queue. The five methods listed above all exist in the blocking queue, but it should be noted that these five methods are synchronized in the blocking queue. In addition to this, blocking queues provide 4 other very useful methods:
put(E e)
take()
offer(E e,long timeout, TimeUnit unit)
poll(long timeout, TimeUnit unit)
The put method is used to store elements to the end of the queue, and if the queue is full, wait;
The take method is used to fetch elements from the head of the queue, and wait if the queue is empty;
The offer method is used to store elements to the end of the queue. If the queue is full, it will wait for a certain time. When the time limit is reached, if the insertion has not been successful, it will return false; otherwise, it will return true;
The poll method is used to fetch elements from the head of the queue. If the queue is empty, it will wait for a certain period of time. When the time limit is reached, if it is fetched, it will return null; otherwise, it will return the fetched element;
3. Implementation principle of blocking queue
The methods commonly used in non-blocking queues and blocking queues have been discussed earlier. Let’s discuss the implementation principles of blocking queues. This article takes ArrayBlockingQueue as an example. The implementation principles of other blocking queues may be somewhat different from ArrayBlockingQueue, but the general idea should be similar. Interested Friends can view the implementation source code of other blocking queues by themselves.
First look at several member variables in the ArrayBlockingQueue class:
public class ArrayBlockingQueue<E> extends AbstractQueue<E>
implements BlockingQueue<E>, java.io.Serializable {
private static final long serialVersionUID = -817911632652898426L;
/** The queued items */
private final E[] items;
/** items index for next take, poll or remove */
private int takeIndex;
/** items index for next put, offer, or add. */
private int putIndex;
/** Number of items in the queue */
private int count;
/*
* Concurrency control uses the classic two-condition algorithm
* found in any textbook.
*/
/** Main lock guarding all access */
private final ReentrantLock lock;
/** Condition for waiting takes */
private final Condition notEmpty;
/** Condition for waiting puts */
private final Condition notFull;
}
It can be seen that what is used to store elements in ArrayBlockingQueue is actually an array, takeIndex and putIndex represent the subscripts of the first and last elements of the queue respectively, and count represents the number of elements in the queue.
lock is a reentrant lock, notEmpty and notFull are wait conditions.
Let's take a look at the constructor of ArrayBlockingQueue. There are three overloaded versions of the constructor:
public ArrayBlockingQueue(int capacity) {
}
public ArrayBlockingQueue(int capacity, boolean fair) {
}
public ArrayBlockingQueue(int capacity, boolean fair,
Collection<? extends E> c) {
}
The first constructor has only one parameter to specify the capacity, the second constructor can specify the capacity and fairness, and the third constructor can specify the capacity, fairness and initialize with another set.
Then look at the implementation of its two key methods: put() and take():
public void put(E e) throws InterruptedException {
if (e == null) throw new NullPointerException();
final E[] items = this.items;
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
try {
try {
while (count == items.length)
notFull.await();
} catch (InterruptedException ie) {
notFull.signal(); // propagate to non-interrupted thread
throw ie;
}
insert(e);
} finally {
lock.unlock();
}
}
It can be seen from the implementation of the put method that it first acquires the lock and acquires an interruptible lock, and then judges whether the current number of elements is equal to the length of the array. If they are equal, call notFull.await() to wait. If captured To interrupt exception, wake up the thread and throw the exception.
When awakened by other threads, elements are inserted through the insert(e) method, and finally unlocked.
Let's take a look at the implementation of the insert method:
private void insert(E x) {
items [putIndex] = x;
putIndex = inc (putIndex);
++count;
notEmpty.signal();
}
It is a private method. After the insertion is successful, it wakes up the thread waiting to fetch the element through notEmpty.
Here is the implementation of the take() method:
public E take() throws InterruptedException {
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
try {
try {
while (count == 0)
notEmpty.await();
} catch (InterruptedException ie) {
notEmpty.signal(); // propagate to non-interrupted thread
throw ie;
}
E x = extract();
return x;
} finally {
lock.unlock();
}
}
It is very similar to the implementation of the put method, except that the put method waits for the notFull signal, and the take method waits for the notEmpty signal. In the take method, if the element can be taken, the element is obtained by the extract method. The following is the implementation of the extract method:
private E extract() {
final E[] items = this.items;
E x = items[takeIndex];
items[takeIndex] = null;
takeIndex = inc(takeIndex);
--count;
notFull.signal();
return x;
}
Similar to the insert method.
In fact, everyone should understand the implementation principle of blocking queue from here. In fact, it is similar to our idea of implementing producer-consumer with Object.wait(), Object.notify() and non-blocking queue, but it combines these tasks together Integrated into the blocking queue implementation.
4. Examples and usage scenarios
The following first uses Object.wait() and Object.notify(), non-blocking queue to implement the producer-consumer mode:
public class Test {
private int queueSize = 10;
private PriorityQueue<Integer> queue = new PriorityQueue<Integer>(queueSize);
public static void main(String[] args) {
Test test = new Test();
Producer producer = test.new Producer();
Consumer consumer = test.new Consumer();
producer.start();
consumer.start();
}
class Consumer extends Thread{
@Override
public void run() {
consume();
}
private void consume() {
while(true){
synchronized (queue) {
while(queue.size() == 0){
try {
System.out.println("Queue empty, waiting for data");
queue.wait();
} catch (InterruptedException e) {
e.printStackTrace ();
queue.notify();
}
}
queue.poll(); //Remove the first element of the queue each time
queue.notify();
System.out.println("Remove an element from the queue, the queue remains "+queue.size()+" elements");
}
}
}
}
class Producer extends Thread{
@Override
public void run() {
produce();
}
private void produce() {
while(true){
synchronized (queue) {
while(queue.size() == queueSize){
try {
System.out.println("Queue is full, waiting for free space");
queue.wait();
} catch (InterruptedException e) {
e.printStackTrace ();
queue.notify();
}
}
queue.offer(1); //Insert one element at a time
queue.notify();
System.out.println("Insert an element into the queue, the remaining space in the queue: "+(queueSize-queue.size()));
}
}
}
}
}
This is the classic producer-consumer pattern, implemented through blocking queues and Object.wait() and Object.notify(), which are mainly used to implement inter-thread communication.
The specific inter-thread communication methods (the use of wait and notify) will be described in the subsequent question chapters.
Here is the producer-consumer pattern implemented using blocking queues:
public class Test {
private int queueSize = 10;
private ArrayBlockingQueue<Integer> queue = new ArrayBlockingQueue<Integer>(queueSize);
public static void main(String[] args) {
Test test = new Test();
Producer producer = test.new Producer();
Consumer consumer = test.new Consumer();
producer.start();
consumer.start();
}
class Consumer extends Thread{
@Override
public void run() {
consume();
}
private void consume() {
while(true){
try {
queue.take();
System.out.println("Remove an element from the queue, the queue remains "+queue.size()+" elements");
} catch (InterruptedException e) {
e.printStackTrace ();
}
}
}
}
class Producer extends Thread{
@Override
public void run() {
produce();
}
private void produce() {
while(true){
try {
queue.put(1);
System.out.println("Insert an element into the queue, the remaining space in the queue: "+(queueSize-queue.size()));
} catch (InterruptedException e) {
e.printStackTrace ();
}
}
}
}
}
Have you found that it is much simpler to use blocking queue code, and you don't need to think about synchronization and inter-thread communication separately.
In concurrent programming, it is generally recommended to use blocking queues, so that the implementation can try to avoid unexpected errors in the program.
The most classic scenario of using blocking queues is the reading and parsing of socket client data. The thread that reads the data keeps putting data into the queue, and then the parsing thread keeps fetching data from the queue for parsing. There are other similar scenarios, and blocking queues can be used as long as they conform to the producer-consumer model.
Reprinted from: http://www.cnblogs.com/dolphin0520/p/3932906.html