【linux】进行间通信——共享内存+消息队列+信号量

进程间通信方式目前我们已经学了匿名管道，命名管道。让两个独立的进程通信，前提是看到同一份资源。匿名管道适用于血缘关系的进程，一个打开写端一个打开读端实现的。命名管道适用于完全独立的进程，打开同一份文件实现的。

接下来我们看看剩下的实现进程间通信的方式。

1.共享内存

1.1共享内存的原理

可执行程序加载到内存，OS会对进程进行管理。进程的内核结构是独立的，加载到物理地址的地址也独立，因此进程具有独立性，互不影响。

那共享内存是如何实现两个独立的进程进行通信的呢？

1.申请一块空间（用户给OS发信号，然后OS去申请）

2.将创建好的内存经过页表映射到进程地址空间中的一段区域（将这块区域的起始地址返回给用户，用户通过访问这里的起始地址方式来进行对这块区域的访问）

3.未来不想通信了
a.取消进程和内存的映射关系
b.释放内存

上面就是共享内存的原理。

这块申请的内存—>共享内存
进程和内存建立映射关系过程—>进程和共享内存挂接
取消进程和内存的映射关系—>去关联
释放内存—>释放共享内存

如何理解上面这个过程？

在内存上申请空间，然后把地址返回，是不是像C/C++空间的申请，如malloc函数，底层都是一样的先去物理内存申请空间，然后经过页表映射到进程地址空间，最后把这块空间的起始地址返回用户。虽然过程都是一样，但是malloc申请的空间没有办法让另一个进程看见，因为这块空间是在堆上申请的，OS并没有专门为malloc这样的机制和其他进程建立映射关系策略。

a.进程间通信，是专门设计的，用来IPC
b.共享内存是一种通信方式，所有想通信的进程，都可以用
c.OS中一定可能会同时存在很多的共享内存

1.2共享内存的概念

通过让不同的进程，看到同一个内存的方式：共享内存。

1.3接口的认识

1.创建共享内存。

size 要申请多大的内存空间

shmflg 常用参数。

看到这里这个参数是不是和open接口有点类似，大写的宏。

shmflg是一个标志位。

IPC_CREAT：如果想创建的共享内存不存在，就创建，如果存在就获取
IPC_EXCL：无法单独使用。
IPC_CREAT | IPC_EXCL：如果不存在就创建，如果存在就出错返回。用户创建共享内存如果成功，一定是一个新的共享内存。

创建共享内存非常容易，那如果保证进程看到的是同一块共享内存呢？
key用来保证。

key是什么不重要，能进行唯一性标识最重要。

将路径名和项目标识符转换为key。

随便写个路径和项目id。经过算法转换成key。
两个进程传一样参数，能保证是同一个key，因此可以在系统中找到同一个内存。

返回值。成功是返回共享内存的标识符，失败返回-1。

再来理解key_t key

OS中一定可能会同时存在很多的共享内存。
OS也要对共享内存进行管理—>先描述，在组织。

申请一块空间—>共享内存=物理内存块+共享内存的相关属性

key是什么不重要，能进行唯一标识最重要。
创建共享内存的时候，key能保证共享内存在系统中的唯一性。两个进程如何看到同一份资源，只要另一个进程也看到同一个key。

key在那？
key在共享内存的相关属性这个结构体里。

创建共享内存把key传到shmget，本质上是把key设置进创建好的共享内存的某个属性里，另一个进程获取时，查这么多的共享内存，不是查共享内的物理内存块，而是去遍历共享内存对于的相关属性去查找key。

key传到shmget，设置到共享内存属性中，用来表示共享内存在内核中的唯一性。

返回值返回的共享内存的标识符取名shmid。

shmid vs key的关系
shmid是为了用户去访问共享内存的。就像fd vs inode的关系。

2.共享内存和进程关联

shmid：和哪一个共享内存关联
shmaddr：把共享内存映射到地址空间的那一块区域
shmflg：与读写权限有关，默认设置为0

成功是返回的是对应进程地址空间的起始地址，失败返回-1。

3.删除共享内存之前要先去关联

将共享内存从当前调用这个函数的进程地址空间进行卸装

shmaddr：进程地址空间的首地址

4.删除共享内存

删除共享内存接口是shmctl，本质上控制共享内存，不过常用的是删除选项。

shmid：控制哪一个共享内存
cmd：做什么控制，常用选项IPC_RMID
buf：如果不想获得共享内存的属性可以设置nullptr，不然就传一个对象接收共享内存部分属性信息。

1.4实操

comm.hpp

将写端和读端用的代码封装起来。

#include<iostream>
#include<cerrno>
#include<cstring>
#include<sys/types.h>
#include<sys/ipc.h>
#include<sys/shm.h>

#define PATHNAME "."
#define PROJ_JD  "0x11"

#define MAX_SIZE 4096

key_t GetKey()
{
    
    
    key_t k=ftok(PATHNAME,PROJ_JD);//获得唯一标识key
    if(k < 0)
    {
    
    
        //cin cout cerr->stdin stdout stderr(默认打开的三个文件)-(fd)>0,1,2->键盘，显示器，显示器
        cerr<<errno<<":"<<strerror(errno)<<endl;
        exit(1);
    }
    return k;
}

//获得共享内存表示符shmid
int getShmHelper(int key,int shmflg)
{
    
    
    int shmid=shemget(key,MAX_SIZE,shmflg);
    if(shmid < 0)
    {
    
    
        
        cerr<<errno<<":"<<strerror(errno)<<endl;
        exit(1);
    }
    return shmid;
}

//写端创建共享内存
int CreateShm(key_t key)
{
    
    
	//这里运行时会报错。下面再看运行结果有解决方法
    return getShmHelper(key,IPC_CREAT|IPC_EXCL);
}

//读端获取共享内存
int GetShm()
{
    
    
    return getShmHelper(key,IPC_CREAT);
}

看运行结果写端和读端的key是一样的，shmid也是一样的。

当我再次执行一样的操作，发现不能再创建共享内存了。显示已经存在了。可是我已经退出进程了啊。OS不会帮我自动关闭吗。

共享内存的生命周期是随操作系统的，不是随进程的

查看IPC资源

ipcs查看IPC资源

ipcs -m 查看共享内存

ipcs -q 查看队列

ipcs -s 查信号量

ipcrm -m shmid 删除共享内存

代码删除共享内存

//删除共享内存
void DelShm(int shmid)
{
    
    
    if(shmctl(shmid,IPC_RMID,nullptr) == -1)
    {
    
    
        cerr<<errno<<":"<<strerror(errno)<<endl;
    }

}

通信之前需要关联（挂接：将共享内存经页表映射到进程地址空间）

void* attachShm(int shimid)
{
    
    
    void* mem=shmat(shimid,nullptr,0);//linux 64位机器指针大小位9
    if((long long)men == -1L)//1L代表是长整型
    {
    
    
        cerr<<errno<<":"<<strerror(errno)<<endl;
        exit(1);
    }
    return men;
}

删除共享内存之前需要去关联

void detachShm(const void* adder)
{
    
    
    if(shmdt(adder) == -1)
    {
    
    
        cerr<<errno<<":"<<strerror(errno)<<endl;
        exit(1);
    }
}

完整代码如下

#include<iostream>
#include<cerrno>
#include<cstring>
#include<sys/types.h>
#include<sys/ipc.h>
#include<sys/shm.h>
#include<unistd.h>

using namespace std;

#define PATHNAME "."
#define PROJ_JD  0x11

#define MAX_SIZE 4096


key_t GetKey()
{
    
    
    key_t k=ftok(PATHNAME,PROJ_JD);
    if(k < 0)
    {
    
    
        //cin cout cerr->stdin stdout stderr(默认打开的三个文件)-(fd)>0,1,2->键盘，显示器，显示器
        cerr<<errno<<":"<<strerror(errno)<<endl;
        exit(1);
    }
    return k;
}

//获得共享内存表示符shmid
int getShmHelper(int key,int flage)
{
    
    
    int shmid=shmget(key,MAX_SIZE,flage);
    if(shmid < 0)
    {
    
        
        cerr<<errno<<":"<<strerror(errno)<<endl;
        exit(1);
    }
    return shmid;
}

//写端创建共享内存
int CreateShm(key_t key)
{
    
    
	//这里运行时会报错。下面再看运行结果有解决方法
    return getShmHelper(key,IPC_CREAT|IPC_EXCL);
}

//读端获取共享内存
int GetShm(key_t key)
{
    
    
    return getShmHelper(key,IPC_CREAT);
}

//关联
void* attachShm(int shimid)
{
    
    
    void* mem=shmat(shimid,nullptr,0);//linux 64位机器指针大小位9
    if((long long)mem == -1L)//1L代表是长整型
    {
    
    
        cerr<<errno<<":"<<strerror(errno)<<endl;
        exit(1);
    }
    return mem;
}

//去关联
void detachShm(const void* adder)
{
    
    
    if(shmdt(adder) == -1)
    {
    
    
        cerr<<errno<<":"<<strerror(errno)<<endl;
        exit(1);
    }
}

//删除共享内存
void DelShm(int shmid)
{
    
    
    if(shmctl(shmid,IPC_RMID,nullptr) == -1)
    {
    
    
        cerr<<errno<<":"<<strerror(errno)<<endl;
    }

}

service.cc （写）

#include"comm.hpp"

int main()
{
    
    
    key_t key=GetKey();
    printf("key->%d\n",key);
    int shmid=CreateShm(key);
    printf("shimid->%d\n",shmid);

    //关联
    char*start=(char*)attachShm(shmid);//我想将这个地址认为是一字符串;
    printf("attach success, address start: %p\n", start);
    
    //写
    while(true)
    {
    
    

    }

    //删除
    DelShm(shmid);

    return 0;
}

共享内存没有read和write这样的接口。
写—>直接把内容写到这块内存
读—>直接打印这块内存

以往我们可能是这样的写法，对于往内存中写有点麻烦了。

    //写
    char buffer[1024];
    const char* messge="hello clint,我是另一个进程,我正在和你通信";
    int id=getpid();
    int cnt=0;
    while(true)
    {
    
    
        //以往我们的做法
        snprintf(buffer,sizeof buffer,"%s([%d]->[%d])",messge,cnt++,id);
        memcpy(start,buffer,strlen(buffer)+1);
    }

看这个函数，我们可以直接把内容写到内存。

完整代码如下

#include"comm.hpp"

int main()
{
    
    
    key_t key=GetKey();
    printf("key->%d\n",key);
    int shmid=CreateShm(key);
    printf("shimid->%d\n",shmid);

    //关联
    char* start=(char*)attachShm(shmid);//我想将这个地址认为是一字符串;
    printf("attach success, address start: %p\n", start);
    
    //写
    char buffer[1024];
    const char* messge="hello clint,我是另一个进程,我正在和你通信";
    int id=getpid();
    int cnt=0;
    while(true)
    {
    
    
        snprintf(start,MAX_SIZE,"%s([%d]->[%d])",messge,cnt++,id);
        //以往我们的做法
        // snprintf(buffer,sizeof buffer,"%s([%d]->[%d])",messge,cnt++,id));
        // memcpy(start,buffer,strlen(buffer)+1);
    }

    //去关联
    detachShm(start);

    //删除
    DelShm(shmid);

    return 0;
}

clint.cc （读）

#include"comm.hpp"

int main()
{
    
    
    key_t key=GetKey();
    printf("key->%d\n",key);
    int shmid=GetShm(key);
    printf("shimid->%d\n",shmid);

    char* start=(char*)attachShm(shmid);
    printf("attach success, address start: %p\n", start);

    while(true)
    {
    
    
         printf("service say : %s\n", start);
    }

    return 0;
}

运行时报了这个没有权限的错误。
这是因为再创建共享内存的时候没有给权限。

补上权限即可

看运行结果，虽然两个进程进行了通信，但是这个通信有点问题。
共享内存不像管道那样，阻塞等待，而是一直在读。

1.5共享内存的总结

共享内存的特点：

1. The life cycle of shared memory depends on the OS, not the process.
2. The fastest communication speed among all processes (can greatly reduce the number of data copies) -> Advantages.

If the same code is implemented using pipelines, considering pipelines and shared memory, keyboard input and monitor output, how many data copies are there in the shared memory? What about pipelines?

3. It does not allow me to perform synchronization and mutual exclusion operations, and does not provide any protection for the data---->Disadvantages

1.6 Kernel structure of shared memory

Pass a struct shmid_de object to this interface and you will see some information about the shared memory.

#include"comm.hpp"

int main()
{
    
    
    key_t key=GetKey();
    printf("key->%d\n",key);
    int shmid=GetShm(key);
    printf("shimid->%d\n",shmid);

    char* start=(char*)attachShm(shmid);
    printf("attach success, address start: %p\n", start);

    while(true)
    {
    
    
         printf("service say : %s\n", start);
         struct shmid_ds ds;
         shmctl(shmid,IPC_STAT,&ds);
         printf("获得属性: size :%d,link :%d\n",ds.shm_segsz,ds.shm_nattch);
         sleep(1);
    }

    return 0;
}

We do gain some properties of the shared kernel.

Our key is in the first variable of the struct shmid_ds structure.

#include"comm.hpp"

int main()
{
    
    
    key_t key=GetKey();
    printf("key->%d\n",key);
    int shmid=GetShm(key);
    printf("shimid->%d\n",shmid);

    char* start=(char*)attachShm(shmid);
    printf("attach success, address start: %p\n", start);

    while(true)
    {
    
    
         printf("service say : %s\n", start);
         struct shmid_ds ds;
         shmctl(shmid,IPC_STAT,&ds);
         printf("获得属性: size :%d,link :%d,key :%d\n",\
                ds.shm_segsz,ds.shm_nattch,ds.shm_perm.__key);
         sleep(1);
    }

    return 0;
}

It is generally recommended that the shared memory size be an integer multiple of 4KB (4096)
The system allocates shared memory in single multiples of 4KB! ------The basic unit of memory division into memory blocks, page

Although the requested size is 4097, the actual size given to you by the kernel is 40962, and the kernel will round it up.
Although the kernel gives you 40962, please note that what the kernel gives you is different from what you can use.

2.Message queue

As mentioned before, System V IPC -> focuses on local communication, which is now outdated. Shared memory is still worth learning. For the rest, we can just take a look at the principles and interfaces.

2.1 Principle

Processes A and B can be read and write segments for each other. One end puts data into the queue and the other end gets it.
Then how to ensure that the two processes will not get their own, but get the other party's.

There is a type in the queue kernel data structure, which is used to identify who sent this information. If it is not your own, you will not take it.

2.2 Interface

Create a message queue

See that these parameters are very similar to the parameters of shared memory.

The message queue identifier is returned on success, and -1 is returned on failure.

Put data into the message queue

msgp: sent data
msgsz: data size
msgflg: default is 0

Receive data

msgp: put the received data here
msgsz: size
msgtyp: type

delete queue

These interfaces are similar to shared kernels.

3. Signal amount

Here are some additional concepts about semaphores, also for later learning.

3.1 What is a semaphore?

is essentially a counter, which is usually used to indicate the number of resources in public resources.

A semaphore is essentially a counter. Can a global variable be directly set to act as a counter?

No, just like setting a global variable in an anonymous pipe, because of copy-on-write, the parent and child processes will not see the same global variable at all.

Processes must see the same resource before they can communicate.
Public resources: resources that can be accessed by multiple processes at the same time.

The two parties communicating through the pipeline noticed that the writing side was not writing, the reading side was blocking and waiting, the reading side was not reading, and the writing side was blocking and waiting after writing, etc. As for the shared memory method, no matter whether the writing side was writing or not, Finished, the reader keeps reading. Suppose you write a piece of data, but the data is read before it is finished. This creates a problem, the data is not protected in shared memory.

Accessing unprotected public resources leads to data inconsistency.

Let’s walk through this process.

Why should different processes see the same resource?
Because I want to communicate and achieve collaboration between processes, but the processes are independent, so I need to let the processes see the same resource first. The method was proposed, and then a new problem was also introduced------>Data inconsistency problem.

The common resources we will protect in the future: critical resources
Most of the resources of the process are independent.

Resources (memory, files, network) need to be used. How are they used by the process?
The process must have corresponding code to access this part of critical resources. This code is a critical section, and other codes are called non-critical sections.

How to protect public resources?
Mutual exclusion and synchronization
Mutual exclusion: When two processes want to access a common resource, we can only allow one process to have complete access.
Synchronization, let’s talk about multi-threading.

There is another set of concepts that are harder to understand.
If I want to write a piece of data to the buffer, I must finish writing the data before you can read it. You can't read it until I finish writing it. For me, when I am writing, I either don’t write, or I finish writing before it is meaningful to you.
This kind of either don’t do it, or do it as soon as you do it, this situation of two states: atomicity.

Suppose Zhang San goes to the bank to transfer 200 to Li Si
Zhang San 1000 Li Si 1000
1000-200 1000+200
However, there is a problem with the network and the transfer fails. The bank cannot just ignore it. It must return the 200 to Zhang San and keep it intact.
There are only two states of transfer for us, either not to transfer, or to complete the transfer if you want to. Although there may be an intermediate process, the final result is either not to transfer, or to complete the transfer if you want. It won’t say that the transfer is in progress.
This is all we have about atomicity.

The above is mainly to illustrate that multi-processes and multi-threads can be used for atomic mutual exclusion and synchronization. Semaphores are one of the solutions.

3.2 Why semaphores are needed

for example

When I go to the cinema to watch a movie, do I only need to sit on the seat to make this seat mine?

No, after buying the ticket (ticket number, seat number), the seat belongs to me at that time.

The essence of buying tickets for watching movies: a mechanism for reserving seats in the screening hall.

When we want a certain resource, we can make a reservation. Cinemas are equivalent to shared resources.

When each process wants to access certain public resources, it must first apply for a semaphore. A successful application is equivalent to booking a certain part of the shared resource before it is allowed to access this small resource. If the application fails, the basic resource is not allowed to enter the share. resources, thereby achieving the purpose of protecting shared resources and other infrastructure.

Before accessing public resources, all processes must first apply for the sem semaphore----> The prerequisite for applying for the sem semaphore is that all processes must first see the same semaphore----> ;The semaphore itself is a public resource---->
Should the semaphore also protect its own security? Woolen cloth? (++, - - operation)----> The semaphore must protect the safety of this operation, ++, - - operation is atomic! ! !

++, - - is what we call PV operation

The essence of a semaphore is a counter. This counter can be seen by multiple processes in a multi-process environment. It must be matched with two operations, P operation and V operation, to allow the process to access our shared resources.

If the initial value of the semaphore is 1, it means that the shared resource is used as a whole. If one process applies successfully, other processes cannot apply ------->Mutually exclusive.

Semaphore that provides mutual exclusion function---->Binary semaphore.

There may be a problem here. If a process successfully applies for a semaphore, it is eligible to access shared resources, but it is not known which resource sub-part it will access. There may be problems with multiple processes accessing the same resource (that is, buying What should I do if I have a ticket (I have the ticket number but not the seat number)?

This part is actually for us to confirm when different processes access that resource when writing code.

3.3 Interface

Get semaphore

nsems: I want to apply for several semaphores

A successful application returns an identifier of the semaphore set. Returns -1 on failure.

Delete the semaphore, or get the semaphore properties

semnum: Maybe you apply for many semaphores. This is the subscript corresponding to the semaphore. To apply for a semaphore, the subscript is 0. If you apply for 10 semaphores, suppose I want to operate on the 10th semaphore, the subscript is 9.

cmd: Delete or obtain semaphore-related properties.

Perform PV operations on semaphores

Perform corresponding operations on the specified semaphore

The struct sembuf structure has three variables

sem_num: Which of the multiple applied semaphores is to be operated on.
sem_op: Generally there are only two values, one is 1 representing ++, V operation. One is -1 represents - -, P operation.
sem_flg: Default is 0

nspos: There are several such structures.

Allows simultaneous PV operations on multiple semaphores.

4.How IPC resources are organized

We noticed that the interfaces of shared memory, message queues, and semaphores are very similar. There are structures like struct...id_ds, and the first field corresponding to describing the IPC attribute resource is the same.

One detail can be seen from these three commonalities. These three are called System V standard inter-process communication.
The so-called standard means that the methods used by everyone, interface design, and data structure settings must comply with certain standards.

So how does the OS manage these (such as applying for shared memory, message queues, and semaphores)?

The OS does not manage them themselves, but first describes the organization and manages the kernel structure of matching related resources.

Because the first field of the attributes in the current three methods is the same.

The operating system can maintain an array of pointers in the kernel.

The address of the first member of the structure is numerically equal to the address of the structure object itself! !

The OS has a corresponding way to know what structure pointer to force into.

This is how the OS organizes IPC resources.