Table of contents

Producer-consumer model based on circular queue

words written in front

This article mainly introduces POSIX semaphores and the use of some interfaces, and then codes and designs a production-consumer model based on ring queues to use these interfaces.

When explaining POSIX semaphores, you first need to have a certain understanding of semaphores. You can read my article: systemV semaphores . In the front of the article, I explained in detail what semaphores are and their understanding.

What is a POSIX semaphore

POSIX semaphore (POSIX semaphore) is a thread synchronization mechanism used to manage concurrent access to shared resources. POSIX semaphores are a set of functions and data types defined based on the POSIX standard, aiming to provide cross-platform thread synchronization capabilities.

POSIX semaphores allow threads to acquire a semaphore before accessing shared resources , and control access to resources by increasing or decreasing the value of the semaphore . When the semaphore value is greater than zero , the thread can acquire the resource and continue executing. When the semaphore value is zero, the thread will be blocked until other threads release the resource and increment the semaphore 's value. This effectively achieves mutually exclusive access to shared resources and synchronization between threads.

Use of POSIX semaphores

sem_init(): Used to initialize a semaphore object.

The function prototype is as follows:
int sem_init(sem_t *sem, int pshared, unsigned int value);
pshared: 0 means shared between threads , non-zero means shared between processes

value: the initial value of the semaphore

sem_destroy(): Used to destroy a semaphore object.

The function prototype is as follows:
int sem_destroy(sem_t *sem)

sem_wait(): Attempts to acquire a semaphore, if the semaphore value is greater than zero, decrement it and continue execution; otherwise, the thread will be blocked.

The function prototype is as follows:
int sem_wait(sem_t *sem); //P()

sem_post(): Release a semaphore, increment its value, and wake up threads that may be waiting for the semaphore.

The function prototype is as follows:
int sem_post(sem_t *sem);//V()

sem_trywait(): sem_wait() Similar to, but instead of blocking the thread when attempting to acquire the semaphore, returns immediately.
sem_timedwait(): sem_wait() Similar to , but you can set the timeout period, if the semaphore cannot be obtained within the timeout period, an error will be returned.

The above two functions can be understood.

Producer-consumer model based on circular queue

This is equivalent to changing the trading place, from a blocking queue to a ring queue.

Here are a few strategies for using circular queues:

The ring queue adopts array simulation, and uses modulo operation to simulate the ring characteristics

But there is a problem with the above design, assuming that head produces and tail consumes; that is, when head and tail overlap, we don't know whether head == tail or tail == head, that is, we don't know whether the ring queue is empty or not . Full. Therefore, it can be judged whether it is full or empty by adding a counter or a flag bit. In addition , an empty position can also be reserved as a full state. For this principle, you can go to the Internet to find relevant knowledge about the ring queue . This empty position just separates the head and tail.

But now there is a semaphore counter , so the process of using ring queue synchronization between threads can also be easily realized.

The overall code design is as follows:

First design a ring queue RingQueue with a class, and encapsulate some interfaces push() and pop(), the member variable is a vector array (using array simulation to realize the ring queue ), int num_ is used to indicate the size of the ring queue, c_step indicates the consumer The subscript of p_step indicates the subscript of the producer, and then there are two semaphores space_sem_ and data_sem_ , which respectively represent the space resource semaphore and the data resource semaphore. At the beginning, there is no data, and all space is available , so we set the space resource semaphore space_sem_ to the length n of the ring queue , and data_sem_ to 0 , indicating that there is no data.

The data type of the semaphore is sem_t , but for convenience , I made a package for the semaphore, class Sem, which contains the initialization of the semaphore, p() and v() operations, etc., and then the types of the above two semaphores Just use Sem directly.

Then in the RingQueue class, the implementation logic of the two interfaces push and pop is as follows:

push(): This is used by the producer . The producer focuses on space resources . If there is space, it will produce, and if there is no space, it will stop. After production, the space resource semaphore space_sem_-1 , but the data resource semaphore data_sem_+1 , and then write the corresponding data in the corresponding position each time. Remember the length of the module array, because the logical structure is a circular queue.
pop(): This is used by consumers . Consumers are concerned about data resources . If there is data, they will consume. If there is no data, they will not be able to consume. After consumption, the data resource semaphore data_sem_-1, but the space resource semaphore space_sem+1, similarly take out the data at the corresponding position.

So the codes of RingQueue class and Sem class are as follows:

Ringqueue.hpp class

#pragma once
#include<iostream>
#include<pthread.h>
#include<vector>
#include "Sem.hpp"
using namespace std;

const int g_default_num = 5;

template<class T>
class RingQueue
{
public:
    //对环形队列进行初始化
    RingQueue(int default_num = g_default_num)
    :ring_queue_(g_default_num),num_(g_default_num),
    c_step(0),p_step(0),
    space_sem_(default_num),data_sem_(0)
    {}
    ~RingQueue()
    {
        
    }
    //生产者：关注空间资源
    void push(const T& in)
    {
        space_sem_.p();
        ring_queue_[p_step++] = in;
        p_step %= num_;
        data_sem_.v();
    }
    //消费者：关注数据资源
    void pop(T* out)
    {
        data_sem_.p();
        *out = ring_queue_[c_step++];
        c_step %= num_;
        space_sem_.v();
    }

private:
    vector<T> ring_queue_;
    int num_;
    int c_step;//消费者下标
    int p_step;//生产者下标
    Sem space_sem_;
    Sem data_sem_;
};

Sem.hpp class

#pragma once
#include "ringQueue.hpp"
#include <semaphore.h>

class Sem
{
public:
    Sem(int val)
    {
        sem_init(&sem_,0,val);
    }
    void p()
    {
        sem_wait(&sem_);
    }
    void v()
    {
        sem_post(&sem_);
    }
    ~Sem()
    {
        sem_destroy(&sem_);
    }
private:
    sem_t sem_;
};

Then we test the code, the test code is as follows, which is almost the same as the test code in the previous section:

#include "ringQueue.hpp"
#include<sys/types.h>
#include<unistd.h>
#include <time.h>
using namespace std;
void* consumer(void* args)
{
    RingQueue<int>* rq = (RingQueue<int>*)args;
    while(true)
    {
        int x;
        //1.从环形队列中拿取数据
        rq->pop(&x);
        //2.进行一定的处理
        cout << "消费: " << x << endl; 

    }
}
void* producter(void* args)
{
    RingQueue<int>* rq = (RingQueue<int>*)args;
    while ((true))
    {
        //1.构建数据或任务对象 -- 一般可以从外部来，不要忽略时间消耗问题
        int x = rand() % 100 + 1;
        cout << "生产: " << x << endl; 
        //2.推送到环形队列中
        rq->push(x);//完成生产的过程
        // sleep(1);

    }
    
}

int main()
{
    srand((unsigned int)time(nullptr) ^ getpid() ^ 12366 );
    RingQueue<int>* rq = new RingQueue<int>();
    pthread_t c,p;
    // rq->debug();
    pthread_create(&c,nullptr,consumer,(void*)rq);
    pthread_create(&p,nullptr,producter,(void*)rq);

    pthread_join(c,nullptr);
    pthread_join(p,nullptr);
    return 0;
}

The code also executes successfully:

The above is a single thread, that is, a single producer and a single consumer.

We also changed it to multi-threaded concurrent execution, which is also the meaning of the production-consumer model.

When multiple threads are executed concurrently, errors may occur if two threads access critical resources at the same time, so it is necessary to add locks before and after critical resources so that only one thread can access critical resources.

But what is the difference between this and single-threaded, all are single-threaded access, what is the meaning of multi-threaded?

First of all, don't narrowly think that putting tasks or data in the trading place is production and consumption. The processing after getting the data or tasks is the most time-consuming . Although they are locked and taken one by one when they are taken, they are processed together when they are processed ! Therefore, the main significance of the production consumer model is reflected in the concurrent processing tasks .

The essence of production: private tasks -> in the public space
The essence of consumption: in public space -> private

The essence of the semaphore is a counter. What is the meaning of the counter?

You can know the status of resources without entering the critical section, and even reduce the judgment inside the critical section.

Apply for locks -> judge critical resources and access -> release locks ---> In essence, we don't know the situation of critical resources, semaphores need to preset resources in advance, and in pv changes, we can know the critical resources externally resource situation.

So we add two locks in the RingQueue class, namely the producer and the consumer:

In the main function, create multiple threads:

After running, you can find different thread production and consumption tasks.

This is the whole content of this chapter, mainly about the POSIX semaphore, which is an implementation of the production consumer model based on the ring queue.

[Linux] POSIX semaphore and production consumer model based on ring queue

words written in front

What is a POSIX semaphore

Use of POSIX semaphores

Producer-consumer model based on circular queue

Guess you like