1. Diagram, step by step decomposition of the theoretical basis

1.1 The theoretical basis of the producer-consumer model using locks + condition variables

When the queue is empty and the queue is full, you need to block the queue. You need to use condition variables to determine whether you need to block the queue.

Thinking 1: What the producer needs for production is to store the produced product in an empty space, and what the consumer needs for consumption is the existence of the product in the queue, so the following is

condition1 : free (vacancy, indicating that the queue is not full, there is a vacancy)
condition2 : full : (with production level, it means that the queue is not empty, there are products)

Pseudo code logic:

When the queue is empty, you need to wait for the product to be in the queue, wait for consumption, and notify the producer to wake up at the same time, please hurry up, you can produce quickly, if the queue definitely meets the conditions of vacancy, you can produce quickly, brother
When the queue is full, you need to wait for the queue to contain vacancies, wait for production, and notify the consumers to wake up at the same time, urging, you hurry up to consume, the queue definitely meets the product conditions, you hurry up and consume it brother
Explain while : Prevent false wakeup: The consumer thread is waiting for the producer thread to produce a product to wake up for consumption, but the producer may just produce a product, but it broadcasts to tell all consumer threads that you wake up and compete for this consumption In fact, in the end, only one consumer can get what they want. The other consumers are equivalent to being deceived. They just woke up, but the product is still empty. What should I do? Go to sleep , so you need to cyclically determine whether you have been pseudo-awakened, and if so, continue to wait

while (Isempty(bq)) {
    Wait(full);
    Notify(free);           
} 

while (IsFull(bq)) {
    Wait(free);
    Notify(full); 
}

Thinking 2: What kind of resources are the products and vacancies in the queue?? They are resources that may be contended by multiple threads at the same time, so what are the resources for write operations that are contended by multiple threads?? They are critical resources , so what is needed is to lock for atomic operations. In the process of multiple threads wanting to write to the blocking queue, the operation of the critical resource of the blocking queue is protected.

//伪代码
Lock();
...  //临界资源处理的逻辑代码块
Unlock();

1.2 The theoretical basis of using semaphores to implement the producer-consumer pattern

In fact, the general logic is similar to the above, except that there are no condition variables and locks here, instead semaphores
Semaphore: My understanding is a critical resource with protection features... What does it mean, a semaphore is a critical resource, the -pthread link library contains a series of encapsulated functions that use this semaphore critical resource.. ...
These function methods themselves contain critical resources for multi-threaded synchronous access (processing mechanism for shared resources)

int sem_init(sem_t *sem, int pshared, unsigned int value); //Initialize the semaphore, pshared : 0 for inter-thread synchronization, 1 for inter-process synchronization, value initialization value

int sem_destroy(sem_t *sem); //Destroy the semaphore

int sem_wait(sem_t *sem); //Consume a critical resource semaphore, if the semaphore val value > 0, directly val -= 1; if the semaphore value == 0, block and wait for it > 0 before consumption

int sem_post(sem_t *sem); //Produce a critical resource semaphore, corresponding to critical resource semaphore += 1, return the resource, and if the previous sem's val == 0 blocks waiting for consumption, this time will be equivalent to waking up the consumer Consumption

//In fact, the semaphore itself is a critical resource shared between producers and consumers. Through its operation, it restricts the orderly and orderly progress of multi-threaded synchronous and exclusive access to this critical resource.

It is mentioned here that synchronous mutual exclusion accesses critical resources. Presumably many people cannot distinguish between synchronous and asynchronous, and mutual exclusion. Here is an illustration:
The above is just my personal opinion. The development of this abstract concept is based on the deepening of the understanding level.
Synchronization events need to wait for each other. For example, when thread 1 writes to a critical resource, other threads need to wait for it to be used up and put it back before using it... So synchronization (hiding a mutual waiting relationship) )

Graphical asynchronous decoupling, using blocking queues:

When explaining asynchronous events, many people will explain from the perspective of function blocking and waiting or function returning directly. In fact, I think the same idea is used. The function does not need to wait and return directly, which actually removes the coupling between the two. Now, I don't need to wait for you to be ready, your sister is ready to take care of me, I'll just go back, you can't restrain me
Synchronous events: On the contrary, I can't return directly if you are not ready, you need to block and wait for you to be ready, such as waiting for the occurrence of IO events

2. Two implementations based on blocking queue in Linux environment (C version)

The form of the queue is actually not important, the important thing is blocking, so I directly use the circular queue to implement it, which is simple and convenient....

2.1 Condition variable + lock implementation version

Analysis of the function method:

#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#include <pthread.h>
#include <signal.h>

//定义阻塞队列
typedef  struct BlockQueue {
	size_t cap;//容量
	size_t size;//当前产品个数
	int front;//队头游标
	int tail;//队尾游标
	int* data; //数据域
	pthread_mutex_t lock;
	pthread_cond_t full;
	pthread_cond_t free;
} BlockQueue;

BlockQueue* bqp;//定义全局方便信号处理时候销毁
void DestroyBlockQueue(BlockQueue* bqp) {
	free(bqp->data);
	pthread_mutex_destroy(&bqp->lock);
	pthread_cond_destroy(&bqp->full);
	pthread_cond_destroy(&bqp->free);
}
//信号处理, 销毁整个阻塞队列。 做结束处理S
void handler(int signo) {
	printf("ByeBye\n");
	DestroyBlockQueue(bqp);
	exit(EXIT_SUCCESS);
}

//初始化阻塞队列
BlockQueue* InitQueue(int n) {
	BlockQueue* bqp = (BlockQueue*)malloc(sizeof(BlockQueue));
	bqp->cap = n;
	bqp->size = 0;
	bqp->front = bqp->tail = 0;
	bqp->data = (int*)malloc(sizeof(int) * n);
	pthread_mutex_init(&bqp->lock, NULL);
	pthread_cond_init(&bqp->free, NULL);
	pthread_cond_init(&bqp->full, NULL);
	return bqp;
}


int IsEmpty(BlockQueue* bqp) {
	return bqp->size == 0;
}

int IsFull(BlockQueue* bqp) {
	return bqp->size == bqp->cap;
}

void WaitConsume(BlockQueue* bqp) {//等待消费, 等队列有产品
	pthread_cond_wait(&bqp->full, &bqp->lock);
}

void WaitProduct(BlockQueue* bqp) {//等待生产, 等队列有空位
	pthread_cond_wait(&bqp->free, &bqp->lock);
}

void NotifyConsume(BlockQueue* bqp) {//通知消费, 队列中有产品了
	pthread_cond_signal(&bqp->full);
}

void NotifyProduct(BlockQueue* bqp) {//通知生产, 队列中有空位了
	pthread_cond_signal(&bqp->free);
}

void Lock(BlockQueue* bqp) {
	pthread_mutex_lock(&bqp->lock);
}

void Unlock(BlockQueue* bqp) {
	pthread_mutex_unlock(&bqp->lock);

}

void Push(BlockQueue* bqp, int val) {
	Lock(bqp);
	while (IsFull(bqp)) {
		WaitProduct(bqp);//没有空位等待生产
		NotifyConsume(bqp);//催促消费
	}
	bqp->data[bqp->tail++] = val;
	bqp->tail %= bqp->cap;
	Unlock(bqp);
	bqp->size += 1;
	NotifyConsume(bqp);//有产品了通知消费
}

void Pop(BlockQueue* bqp, int* popval) {
	Lock(bqp);
	while (IsEmpty(bqp)) {
		WaitConsume(bqp);//没有产品等待消费
		NotifyProduct(bqp);//催促生产
	}
	*popval = bqp->data[bqp->front++];
	bqp->front %= bqp->cap;
	Unlock(bqp);
	bqp->size -= 1;
	NotifyProduct(bqp);//有空位了通知生产
}

void* ConsumeRoutine(void* args) {//消费者线程执行函数
    BlockQueue* bqp = (BlockQueue*)args;
	int popval = 0;
    for ( ;; ) {
        Pop(bqp, &popval);//取出消费产品
	    printf("PopVal is %d, and has %d Products\n", popval, bqp->size); 
        sleep(rand() % 3);
    }
  return (void*)0;
}

void* ProductRoutine(void* args) {//生产者线程执行函数
	BlockQueue* bqp = (BlockQueue*)args;
	int pushval = 0;
    for ( ;;  ) {
	    pushval = rand() % 1024;//随机一个数组当作产品
        Push(bqp, pushval);//装入产品
	    printf("PushVal is %d, and has %d Products\n", pushval, bqp->size); 
        sleep(rand() % 3);
    }
  return (void*)0;
}

int main() {
    signal(SIGINT, handler);//注册信号处理函数
	srand((unsigned int)time(NULL));
	bqp = InitQueue(30);

	pthread_t consume1, consume2, product1, product2; 
	pthread_create(&product1, NULL, ProductRoutine, (void*)bqp);
	pthread_create(&product2, NULL, ProductRoutine, (void*)bqp);
	pthread_create(&consume1, NULL, ConsumeRoutine, (void*)bqp);
	pthread_create(&consume2, NULL, ConsumeRoutine, (void*)bqp);

	pthread_join(product1, NULL);
	pthread_join(product2, NULL);
	pthread_join(consume1, NULL);
	pthread_join(consume2, NULL);
	return 0;
}

2.2 Semaphore implementation version

#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#include <pthread.h>
#include <signal.h>
#include <semaphore.h>

//定义阻塞队列
typedef  struct BlockQueue {
	size_t cap;//容量
	size_t size;//当前产品个数
	int front;//队头游标
	int tail;//队尾游标
	int* data; //数据域
	sem_t data_sem;//产品信号量, 阻塞队列中有多少产品
	sem_t space_sem;//空位信号量. 阻塞队列中有多少空位
} BlockQueue;

BlockQueue* bqp;//定义全局方便信号处理时候销毁
void DestroyBlockQueue(BlockQueue* bqp) {
	free(bqp->data);
	sem_destroy(&bqp->data_sem);
	sem_destroy(&bqp->space_sem);
}
//信号处理, 销毁整个阻塞队列。 做结束处理S
void handler(int signo) {
	printf("ByeBye\n");
	DestroyBlockQueue(bqp);
	exit(EXIT_SUCCESS);
}

//初始化阻塞队列
BlockQueue* InitQueue(int n) {
	BlockQueue* bqp = (BlockQueue*)malloc(sizeof(BlockQueue));
	bqp->cap = n;
	bqp->size = 0;
	bqp->front = bqp->tail = 0;
	bqp->data = (int*)malloc(sizeof(int) * n);
	sem_init(&bqp->data_sem, 0, 0);//初始化产品量0
	sem_init(&bqp->space_sem, 0, n);//初始化空位量为队列容量
	return bqp;
}

void Push(BlockQueue* bqp, int val) {
	sem_wait(&bqp->space_sem);//P 申请一个空位资源
	bqp->data[bqp->tail++] = val;
	bqp->tail %= bqp->cap;
	bqp->size += 1;
	sem_post(&bqp->data_sem); //V 制作或者说产生一个产品资源
}

void Pop(BlockQueue* bqp, int* popval) {
	sem_wait(&bqp->data_sem); //P 申请一个产品
	*popval = bqp->data[bqp->front++];
	bqp->front %= bqp->cap;
	bqp->size -= 1;
	sem_post(&bqp->space_sem);//V 归还一个空位
}

void* ConsumeRoutine(void* args) {//消费者线程执行函数
	BlockQueue* bqp = (BlockQueue*)args;
	int popval = 0;
    for ( ;; ) {
	  Pop(bqp, &popval);//取出消费产品
	  printf("PopVal is %d, and has %d Products\n", popval, bqp->size); 
      sleep(rand() % 3);
    }
  return (void*)0;
}

void* ProductRoutine(void* args) {//生产者线程执行函数
	BlockQueue* bqp = (BlockQueue*)args;
	int pushval = 0;
    for ( ;;  ) {
	  pushval = rand() % 1024;//随机一个数组当作产品
      Push(bqp, pushval);//装入产品
	  printf("PushVal is %d, and has %d Products\n", pushval, bqp->size); 
      sleep(rand() % 3);
   }
  return (void*)0;
}


int main() {
  signal(SIGINT, handler);
	srand((unsigned int)time(NULL));
	bqp = InitQueue(30);
	pthread_t consume1, consume2, product1, product2; 
	pthread_create(&product1, NULL, ProductRoutine, (void*)bqp);
	pthread_create(&product2, NULL, ProductRoutine, (void*)bqp);
	pthread_create(&consume1, NULL, ConsumeRoutine, (void*)bqp);
	pthread_create(&consume2, NULL, ConsumeRoutine, (void*)bqp);

	pthread_join(product1, NULL);
	pthread_join(product2, NULL);
	pthread_join(consume1, NULL);
	pthread_join(consume2, NULL);
	return 0;
}

3. The thread library implementation version generated by the new features of C++11

The first is basic theoretical support, function analysis or something, if you are not clear, you can go to the following blog analysis

https://blog.csdn.net/weixin_53695360/article/details/122969157?spm=1001.2014.3001.5502

#include <iostream>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <queue>
#include <windows.h>
using namespace std;
#define NUM 10

class BlockQueue {
public:
	BlockQueue(int cap = NUM)
		: _cap(cap)
	{}
	void Push(int val) {
		unique_lock<mutex> _lock(mtx);
		while (IsFull()) {
			_free.wait(_lock);//没有空位生产等待
			NotifyConsume();//催促消费
		}
		_q.push(val);//生产产品
		NotifyConsume();//有产品了通知消费
	}

	void Pop(int& popval) {
		unique_lock<mutex> _lock(mtx);
		while (IsEmpty()) {
			_full.wait(_lock);//没有产品消费等待
			NotifyProduct();//催促生产
		}
		popval = _q.front();
		_q.pop();//消费产品
		NotifyProduct();//有空位了通知生产
	}
	
	int Size() {
		return _q.size();
	}
private:
	queue<int> _q;//存储队列
	int _cap;//容量
	mutex mtx;
	condition_variable _full;//存在产品
	condition_variable _free;//存在空位
private:
	void NotifyConsume() {
		_full.notify_one();//通知消费
	}

	void NotifyProduct() {
		_free.notify_one();//通知生产
	}

	bool IsEmpty() {
		return _q.size() == 0;
	}

	bool IsFull() {
		return _q.size() == _cap;
	}
};

void ConsumeRoutine(void* args) {//消费者线程执行函数
	BlockQueue* bqp = (BlockQueue*)args;
	int popval = 0;
	for (;;) {
		bqp->Pop(popval);//取出消费产品
		printf("PopVal is %d, and has %d Products\n", popval, bqp->Size());
		Sleep((rand() % 3) * 100);
	}
}

void ProductRoutine(void* args) {//生产者线程执行函数
	BlockQueue* bqp = (BlockQueue*)args;
	int pushval = 0;
	for (;;) {
		pushval = rand() % 1024;//随机一个数组当作产品
		bqp->Push(pushval);//装入产品
		printf("PushVal is %d, and has %d Products\n", pushval, bqp->Size());
		Sleep((rand() % 3) * 100);
	}
}

int main() {
	srand((unsigned long)time(NULL));
	BlockQueue bq(30);
	thread t1(ProductRoutine, &bq);
	thread t2(ProductRoutine, &bq);
	thread t3(ConsumeRoutine, &bq);
	thread t4(ConsumeRoutine, &bq);

	t1.join();
	t2.join();
	t3.join();
	t4.join();
	return 0;
}

4. Conclusion of this chapter

This chapter mainly introduces the producer-consumer model based on blocking queues. The function of blocking queues is to release the coupling relationship between producers and consumers. Consumers take products from blocking queues, and producers push production to blocking queues. product

The producer-consumer pattern is to solve the problem of strong coupling between producers and consumers through a container. Producers and consumers do not communicate directly with each other, but communicate through blocking queues. Therefore, after producers produce data, they do not need to wait for consumers to process data, but throw them directly to blocking queues. Consumers do not ask producers for data, but Taken directly from the blocking queue, the blocking queue is equivalent to a buffer, balancing the processing capabilities of producers and consumers. This blocking queue is used to decouple producers and consumers.

Then I also introduced the difference between synchronously waiting for an event and asynchronously returning an event that does not wait...

Synchronous blocking IO: After the user's main thread initiates an IO operation, it must suspend and wait for the IO operation to complete before continuing to run.

Synchronous non-blocking IO: The user's main thread returns directly after initiating an IO operation, but it needs to poll whether the loop is IO ready or not. . .

Asynchronous blocking IO: After the user's main thread initiates an IO operation, it still returns directly to do its own thing, but there is no need to poll, other threads will block to help us monitor IO. When satisfied, notify the main thread, big brother IO is ready, you can do things

The last is to use different ways to complete the implementation of the producer-consumer pattern

Producer-consumer mode nanny-level tutorial (blocking queue decoupling) One article helps you from the C language version to the C++ version, from theory to implementation (one article is enough)