What is the producer consumer model
The producer-consumer model is a typical model in multi-threading.
For example: you are a customer and you go to the supermarket to buy ham sausage.
The "you" in this passage are the consumers, so the supplier of ham to the supermarket is the producer. What about the supermarket? Is the supermarket shared by everyone? Everyone has access to supermarkets, so supermarkets here are a critical resource.
Therefore, producers and consumers have three relationships, two roles, and one trading place.
Three relationships:
1. Producers and Producers
2. Consumers and consumers
3. Producers and consumers
Producers are in a competitive relationship with each other because manufacturers compete with each other. Therefore, production and producers aremutually exclusive in a relationship.
Consumers are actually in a competitive relationship with each other, but because there are enough products and consumers consume too slowly, there is no obvious difference. But if there is only the last bottle of mineral water left in the world, will everyone grab it? Therefore, consumers and consumers are actually mutually exclusive relationships.
Producers and consumers are also in a competitive relationship. We, producers and consumers, regard them as two threads, and the supermarket as a critical resource. So do both threads have to access this critical resource? Since both have access to this critical resource, production and consumers are also mutually exclusive. But it’s not just mutual exclusion, because producers have filled the supermarket, do they have to wait for users to consume? In the same way, if the supermarket is empty, do consumers have to wait for producers to supply goods? Therefore, there is another relationship between production and consumers, which is synchronization.
two roles
producers and consumers
a trading place
A critical resource, the producer provides data to the critical resource, and the consumer takes data from the critical resource.
Have you ever noticed that the producer and consumer models look a lot like pipelines? Yes, pipelines are a typical producer and consumer model.
This is a multi-producer and multi-consumer model.
Next we will implement a producer-consumer model based on blocking queues. The blocking queue here is responsible for critical resources. The producer puts data into the blocking queue, and the consumer takes the data out of the blocking queue.
Lock packaging
First we use RAII style locks.
MyLock class
#include<pthread.h>
class MyLock
{
public:
MyLock(pthread_mutex_t* pmtx): _pmtx(pmtx){
}
void Lock(){
pthread_mutex_lock(_pmtx);}
void Unlock() {
pthread_mutex_unlock(_pmtx);}
private:
pthread_mutex_t* _pmtx;
};
LockGuard class
#include<pthread.h>
class LockGuard
{
public:
LockGuard(pthread_mutex_t* pmtx):_mtx(pmtx){
_mtx.Lock();
}
~LockGuard()
{
_mtx.Unlock();
}
private:
MyLock _mtx;
};
The constructor of this class is for locking, and the destructor is for unlocking. So we only need to put the code to create an object of this class and the code of the critical resource together to achieve locking and unlocking. This method can avoid the problem of deadlock caused by sometimes forgetting to write the unlock.
Implementation of blocking queue
Declaration of block_queue class
#include<queue>
#include<pthread.h>
#include<iostream>
#include "Task.hpp"
#include "LockGuard.hpp"
#define DEFAULT_NUM 5
template<class T> //因为不确定阻塞队列放的数据类型， 所以用模板参数
class block_queue
{
private:
size_t _num; //阻塞队列的容量
std::queue<T> _blockqueue; //阻塞队列
pthread_mutex_t _mtx; //锁
pthread_cond_t _full; //条件变量，让生产者在阻塞队列为满时进行等待
pthread_cond_t _empty; //条件变量，让消费者在阻塞队列为空时进行等待
public:
block_queue(size_t num = DEFAULT_NUM); //构造函数
~block_queue(); // 析构
//生产者生产
void Push(const T& task);
// 消费者消费
void Pop(T* out);
private:
//让当前线程在指定的条件变量下等待
void Wait(pthread_cond_t* cond) {
pthread_cond_wait(cond,&_mtx);}
//唤醒指定条件变量下等待的线程
void Wakeup(pthread_cond_t* cond) {
pthread_cond_signal(cond);}
//判断阻塞队列是否满了
bool isfull() {
return _blockqueue.size() == _num;}
//判断阻塞队列是否为空
bool isempty() {
return _blockqueue.size() == 0;}
};
Our blocking queue actually only provides two operations, one is push (producer puts data), and the other is pop (consumer gets data).
Implementation of block_queue class
#define DEFAULT_NUM 5
template<class T>
class block_queue
{
private:
size_t _num;
std::queue<T> _blockqueue;
pthread_mutex_t _mtx;
pthread_cond_t _full;
pthread_cond_t _empty;
public:
block_queue(size_t num = DEFAULT_NUM) : _num(num){
pthread_mutex_init(&_mtx,nullptr);
pthread_cond_init(&_full,nullptr);
pthread_cond_init(&_empty,nullptr);
}
~block_queue()
{
pthread_mutex_destroy(&_mtx);
pthread_cond_destroy(&_full);
pthread_cond_destroy(&_empty);
}
//生产者生产
void Push(const T& task)
{
LockGuard lockguard(&_mtx); //加锁，出了作用域自动解锁
while(isfull()) Wait(&_full); //生产队列已满，生产者在full条件变量下等待
//被唤醒后添加任务到生产队列
_blockqueue.push(task);
printf("%p 生产了一个任务 : %d %c %d\n",pthread_self(),task._x,task._op,task._y); //这是对任务的打印....暂且无视，等Task类实现完后看结果的
Wakeup(&_empty); //唤醒消费者
}
// 消费者消费
void Pop(T* out)
{
LockGuard lockguard(&_mtx) ;//加锁，出了作用域自动解锁
while(isempty()) Wait(&_empty); //生产队列已空，消费者进入等待
//被唤醒后添加任务到生产队列
*out = _blockqueue.front(); //提取任务
_blockqueue.pop(); //队列pop
Wakeup(&_full);
}
private:
void Wait(pthread_cond_t* cond) {
pthread_cond_wait(cond,&_mtx);}
void Wakeup(pthread_cond_t* cond) {
pthread_cond_signal(cond);}
bool isfull() {
return _blockqueue.size() == _num;}
bool isempty() {
return _blockqueue.size() == 0;}
};
Task class implementation
We can put data into the blocking queue, and of course we can also put a task into it. Here we create a task class for addition, subtraction, multiplication, division and modulo operations.
#include <iostream>
class Task{
public:
Task(){
}
Task(int x, char op,int y):_x(x),_op(op),_y(y),_iserror(false){
}
void Runing()
{
int ret = 0;
switch(_op)
{
case '+' : ret = _x + _y; break;
case '-' : ret = _x - _y; break;
case '*' : ret = _x * _y; break;
case '/' :
{
if(_y) ret = _x / _y;
else _iserror = true;
break;
}
case '%' :
{
if(_y) ret = _x % _y;
else _iserror = true;
break;
}
default: _iserror = true;
}
if(_iserror) std::cout << "result error" << std::endl; //如果结果错误打印错误
else std::cout << _x << _op << _y << "=" << ret << std::endl; //如果结果正确打印完整式子
}
public:
int _x; //第一个操作数
char _op; //操作符
int _y; //第二个操作数
bool _iserror; //结果是否错误
};
Main
`
#include "BlockQueue.hpp"
#include <time.h>
#include<unistd.h>
#include<string>
#define CONNUM 5
#define PRODNUM 2
//生产者放任务
void* ProcuderRuning(void* args)
{
wyl::block_queue<wyl::Task>* bq = (wyl::block_queue<wyl::Task>*)args;
while(1)
{
int x = rand() % 10 + 1;
int y = rand()%20;
char op = "+-*/%"[rand() % 5];
bq->Push(wyl::Task(x,op,y)); //往阻塞队列中放任务
}
}
//消费不断拿任务
void* ConsumerRuning(void* args)
{
wyl::block_queue<wyl::Task>* bq = (wyl::block_queue<wyl::Task>*)args;
while(1)
{
wyl::Task t;
bq->Pop(&t); //从阻塞队列中拿任务
printf("%p 消费了一个任务",pthread_self());
t.Runing(); //处理任务
sleep(1); //让消费者不要频繁消费太快，这样阻塞队列满了会等待消费者
}
}
int main()
{
pthread_t con[CONNUM];
pthread_t prod[PRODNUM];
srand((unsigned int)0); //随机数种子
//创造等待队列
wyl::block_queue<wyl::Task>* bq = new wyl::block_queue<wyl::Task>(5);
//创建生产者线程
for(int i = 0 ; i < PRODNUM ; i++)
{
std::string name = "prodcuer ";
name += std::to_string(i+1);
pthread_create(prod + i,nullptr,ProcuderRuning,(void*)bq);
}
//创建消费者线程
for(int i = 0 ; i < CONNUM ; i++)
{
std::string name = "consumer ";
name += std::to_string(i+1);
pthread_create(con + i,nullptr,ConsumerRuning,(void*)bq);
}
//等待线程
for(int i = 0 ; i < PRODNUM ; i++)
{
pthread_join(prod[i],nullptr);
}
for(int i = 0 ; i < CONNUM ; i++)
{
pthread_join(con[i],nullptr);
}
return 0;
}
`
The execution result of the consumer consuming slowly and the producer producing quickly:
The result of the producer producing slowly and the consumer consuming quickly:
We will find that the tasks are executed in an orderly manner. After the producer puts the data, it notifies the consumer to get it. After the consumer gets the data, it notifies the producer to put it back.