Linux knowledge points--interprocess communication (1)

Linux knowledge points - inter-process communication (1)

---

1. Understand interprocess communication

1. The necessity of interprocess communication

Single process, unable to use concurrency capability, let alone realize multi-process coordination, such as transmitting data, synchronizing execution flow, message notification, etc.;

2. Technical background of interprocess communication

(1) The process is independent; the virtual address space + page table guarantees the independence of the process operation (process kernel data structure + process code and data); (2) The communication cost will be relatively high
;

3. Essential understanding of inter-process communication

(1) The premise of inter-process communication is first to let different processes see the same memory space (organized by a specific structure); (2) The
so-called seeing the same space, which process should this space belong to Woolen cloth? Should not belong to any process, but should emphasize sharing;

4. Standards for interprocess communication

• What Linux can provide natively: pipes,
  anonymous pipes;
  named pipes;
• SystemV IPC: multi-process, for stand-alone communication
  shared memory;
  message queue;
  semaphore;
• POSIX IPC: multi-threaded, for network communication

2. Anonymous channels

1. The principle of anonymous channel communication

Pipeline communication is communication between processes through pipes, and a pipe is a piece of memory space that two processes can share;

• The essence of a pipeline is a file:
  (1) The parent process opens a file in the way of reading and writing respectively;
  
  after completion, the file pointers corresponding to the two file descriptors in the parent process point to the same file, one for reading and one for writing, this file is Pipeline;
  (2) fork creates a child process;
  
  after the parent process forks out of the child process, the child process will copy the information of the parent process PCB, so in the file sequence of the child process, there will also be a file pointer corresponding to the same file descriptor Point to the pipeline created by the parent process;
  (3) Both processes close the file descriptors they do not need;
  
  after determining the read and write of the parent and child processes, such as the parent process writes and the child process reads, then close the parent process read and child process write The corresponding fd, at this point, a pipeline is formed;

• Note:
  (1) When creating a child process, only copy the data related to the process, PCB,File related will not be copied, after the copy is completed, the files pointed to by the parent and child processes are the same;
  (2) Inter-process communication is based onMemory(3) The
  | we use on the command line is the pipeline;
  
  the three sleeps are all processes, which are brother processes, and one process processes the data and passes it to the next process through the pipeline;

2. Use of anonymous pipes

pipe function: Create a pipe, which is equivalent to completing the parent process to open a file in read and write mode;

• parameter:
  pipefd[2]：output parameter, expecting to pass this parameter, getfd of the opened file(one fd each for reading and writing),pipefd[0] is the read end, pipefd[1] is the write end;
  Return value:
  success returns 0, failure returns -1;

• makefile：

pipe-use:pipe-use.cpp
	g++ -o $@ $^ -std=c++11
.PHONY:clean
clean:
	rm -f pipe-use

• pipe-use.cpp：

#include<iostream>
#include<string>
#include<cstdio>         //在c++中更好兼容c语言的头文件
#include<cstring>
#include<assert.h>
#include<unistd.h>
#include<sys/types.h>
#include<sys/wait.h>

using namespace std;

int main()
{
    
    
    //1.创建管道
    int pipefd[2] = {
    
    0};
    int n = pipe(pipefd);
    assert(n != -1);
    (void)n;        //debug模式下assert是有效的，而release模式下assert就无效了
                    //(void)n就是将n使用以下，避免release模式下报错

//条件编译，打印出pipefd中的内容
//如果想要执行这段代码，在g++编译选项中加上-DEGUB即可
#ifdef DEBUG
    cout << "pipefd[0]" << pipefd[0] << endl;
    cout << "pipefd[1]" << pipefd[1] << endl;
#endif

    //2.创建子进程
    pid_t id = fork();
    assert(id != -1);
    if(id == 0)
    {
    
    
        //子进程 - 读
        //3.构建单向通信的管道，父进程写入，子进程读取
        //3.1 关闭子进程不需要的fd
        close(pipefd[1]);
        char buffer[1024];
        while(true)
        {
    
    
            ssize_t s = read(pipefd[0], buffer, sizeof(buffer) - 1);//从管道中读取数据
            if(s > 0)
            {
    
    
                buffer[s] = 0;
                cout << "child get a message[" << getpid() << "]father# " << buffer << endl;
            }
        }
        close(pipefd[0]);//关闭子进程读取fd。可以不关闭，因为子进程退出时会关闭其所有fd

        exit(0);
    }

    //父进程
    //3.构建单向通信的管道，父进程写入，子进程读取
    //3.1关闭父进程不需要的fd
    close(pipefd[0]);
    string message = "我是父进程，我正在给你发消息";
    int count = 0;
    char send_buffer[1024];
    while(true)
    {
    
    
        //3.2构建一个变化的字符串
        //sprintf是向字符串中格式化显示内容
        snprintf(send_buffer, sizeof(send_buffer), "%s[%d] : %d",
        message.c_str(), getpid(), count++);
        //3.3写入
        write(pipefd[1], send_buffer, sizeof(send_buffer));
        //3.4sleep
        sleep(1);
    }    

    pid_t ret = waitpid(id, nullptr, 0);
    assert(ret < 0);
    (void)ret;
    
    close(pipefd[1]);

    return 0;
}

• Notice:
  (1) Conditional compilation
  
  If you want to execute this code, add -DEGUB to the g++ compilation option:
  
  （2）snprintf
  
  Safely format and display content into strings;
  
  （3）cstdio & cstring
  
  These two header files are for better compatibility with c language;
  (4) Can defining a global buffer be used for inter-process communication?
  No, because of the existence of copy-on-write, the independence of data must be maintained between the parent and child processes;

3. The characteristics of the pipeline

• (1) The pipeline is used for inter-process communication between processes with blood relationship;
• (2) The pipeline has the function of allowing inter-process collaboration and provides access control;
  The parent process writes data every 1s, and the child process reads it every 1s, but we only set the 1s write when the parent process sends it, and the child process does not set it. This is the access control of the pipeline; if the parent process is
  always Write, the child process reads once in a while:
  

Running result:

We can see that after the parent process fills the pipeline, it is blocked here, waiting for the child process to read;

when the child process reads a certain amount of data, the parent process can continue to write;
Summarize:
a. Writing is fast, reading is slow, and no more writing can be done after the pipeline is full;
b. Writing is slow, reading is fast, and when there is no data in the pipeline, reading must wait;
c. Writing is off, reading will return 0, indicating that the mark has been read The end of the file;
d. Read off, write to continue, and the OS will terminate the process;

• (3) The pipeline provides stream-oriented communication services – byte stream-oriented – protocol
  When the pipeline reads files, it does not read one file at a time, but reads a batch at a time;
• (4) The pipeline is based on the file, the life cycle of the file is with the amount of the process, and the life cycle of the pipeline is with the process
  Let the parent process stop writing after 5s, and observe the child process:
  
  
  running results:
  
  the writing party, fd is not closed, if there is data, read it, and wait if there is no data; the
  writing party closes fd, and the reading party, read will return 0, indicating that the end of the file has been read;
• (5) The pipeline is one-way communication, which is a special case of half-duplex communication
  Half-duplex: can not read and write at the same time, only one side can read and one side can write;

4. Process pool project

The parent process creates four child processes, uses four channels for inter-process communication, dispatches tasks to the child processes, and load balances the stand-alone version;

• makefile：

process-pool:process-pool.cpp
	g++ -o $@ $^ -std=c++11
.PHONY:clean
clean:
	rm -f process-pool

• process-pool.cpp

#include<iostream>
#include<unistd.h>
#include<sys/wait.h>
#include<sys/types.h>
#include<cassert>
#include<vector>
#include<cstdlib>
#include<ctime>
#include"Task.hpp"

#define PROCESS_NUM 5  //子进程数量

using namespace std;

int waitCommand(int waitFd, bool &quit) //如果对方不发，我们就阻塞
{
    
    
    uint32_t command = 0;
    ssize_t s = read(waitFd, &command, sizeof(command));
    if (s == 0)
    {
    
    
        quit = true;
        return -1;
    }
    assert(s == sizeof(uint32_t));
    return command;
}

void sendAndWakeup(pid_t who, int fd, uint32_t command)
{
    
    
    write(fd, &command, sizeof(command));
    cout << "main process: call process " << who << " execute " << desc[command] << " through " << fd << endl;
}

int main()
{
    
    
    //加载方法
    load();
    //保存管道信息
    vector<pair<pid_t, int>> slots; //pid : pipefd
    //先创建多个进程
    for(int i = 0; i < PROCESS_NUM; i++)
    {
    
    
        //创建管道
        int pipefd[2] = {
    
    0};
        int n = pipe(pipefd);
        assert(n != -1);
        (void)n;

        pid_t id = fork();
        assert(id != -1);

        if(id == 0)
        {
    
    
            //child
            //子进程读取
            close(pipefd[1]);
            while(true)
            {
    
    
                //等命令
                bool quit = false;//进程退出判断
                int command = waitCommand(pipefd[0], quit);//如果对方不发，我们就阻塞
                if(quit)//如果父进程停止写入，子进程也停止
                {
    
    
                    break;
                }
                //执行对应的命令
                if(command >= 0 && command < handlerSize())
                {
    
    
                    callbacks[command];
                }
                else
                {
    
    
                    cout << "非法command" << command << endl;
                }
            }
            exit(1);
        }
        // father
        // 父进程写入
        close(pipefd[0]);
        slots.push_back(make_pair(id, pipefd[1])); // 保存管道信息
    }
    // 父进程派发任务（均衡的派发给每一个子进程）
    srand((unsigned long)time(nullptr) ^ getpid() ^ 23323123123L); // 让数据源更随机
    while (true)
    {
    
    
        int slect;
        int command;//任务编号
        cout << "##########################################" << endl;
        cout << "##   1.show functions  2.send command   ##" << endl;
        cout << "##########################################" << endl;
        cout << "Please Slect> ";
        cin >> slect;
        if (slect == 1)
        {
    
    
            showHandler();
        }
        else if (slect == 2)
        {
    
    
            cout << "Enter Your Command> ";
            // 选择任务
            cin >> command;
            // 选择进程
            int choice = rand() % slots.size();
            // 把任务发送给指定的进程
            sendAndWakeup(slots[choice].first, slots[choice].second, command);
        }
        else
        {
    
    
        }

        //关闭fd，所有的子进程都会退出
        for(const auto &slot : slots)
        {
    
    
            close(slot.second);
        }

        //回收所有的子进程信息
        for(const auto &slot : slots)
        {
    
    
            waitpid(slot.first, nullptr, 0);
        }
    }

    return 0;
}

• Task.hpp

#include<iostream>
#include<string>
#include<vector>
#include<unistd.h>
#include<functional>

typedef std::function<void()> func; //定义函数类型，实现函数回调

std::vector<func> callbacks;//存储回调函数
std::unordered_map<int, std::string> desc;

void readMySQL()
{
    
    
    std::cout << "process[" << getpid() << "]执行访问数据库的任务" << std::endl;
}


void executeUrl()
{
    
    
    std::cout << "process[" << getpid() << "]执行解析Url" << std::endl;
}

void cal()
{
    
    
    std::cout << "process[" << getpid() << "]执行加密任务" << std::endl;
}


void save()
{
    
    
    std::cout << "process[" << getpid() << "]执行数据持久化任务" << std::endl;
}

//加载方法和对应的描述
void load()
{
    
    
    desc.insert(callbacks.size(), "readMySQL: 读取数据库");
    callbacks.push_back(readMySQL);

    desc.insert(callbacks.size(), "executeUrl: 进行url解析");
    callbacks.push_back(executeUrl);

    desc.insert(callbacks.size(), "cal: 进行加密计算");
    callbacks.push_back(cal);

    desc.insert(callbacks.size(), "save: 进行文件保存");
    callbacks.push_back(save);

}


void showHandler()
{
    
    
    for(const auto &iter : desc)
    {
    
    
        std::cout << iter.first << "\t" << iter.second << std::endl;
    }
}

int handlerSize()
{
    
    
    return callbacks.size();
}

operation result:

3. Named Pipes

1. The principle of named pipe

Anonymous pipes communicate based on creating copies of process information of sub-processes, and can only be used for blood-related inter-process communication;
while named pipes can realize communication between unrelated processes;

different processes open the same file, The operating system detects the file path, and instead of loading the contents of the file into memory, it points the process to the same file structure;
this is a named pipe, which is a memory-level file, but builds a file on disk Name, to access the file name under the same path when the process accesses;
the named pipe just creates a symbol in the disk, just for the communication parties to see the same resource, and its data is at the memory level and will not be written to the disk input data;

• experiment:
  
  mkfifo creates a named pipe file under the specified path;
  
  after creating a pipe file, p is the pipe file;
  then let two processes communicate through the named pipe:
  
  one process redirects the printed characters to the pipe file, at this time, This process is blocked, which is waiting for other processes to read data from the pipeline;
  
  another process reads data from the pipeline file and prints it, and the writing process can push out the blocking;
  
  continuously write and read;

2. Use of named pipes

mkfifo: Create a named pipe using the system interface:

parameter:
pathname: specify the path;
mode: specify the permission of the pipeline file;
return value: return 0 on success; return -1 on failure;

• makefile

.PHONY:all
all:mutiServer client

mutiServer:mutiServer.cpp
	g++ -o $@ $^ -std=c++11 
client:client.cpp
	g++ -o $@ $^ -std=c++11 

.PHONY:clean
claen:
	rm -f mutiServer client

• comm.hpp

#ifndef _COMM_H_
#define _COMM_H_

#include<iostream>
#include<string>
#include<cstdio>
#include<cstring>
#include<unistd.h>
#include<sys/types.h>
#include<sys/stat.h>
#include<fcntl.h>
#include<sys/wait.h>
#include "Log.hpp"

using namespace std;

#define MODE 0666 //管道文件的权限
#define SIZE 128

string ipcPath = "./fifo.ipc";//管道文件的路径

#endif

• Log.hpp

#ifndef _LOG_H_
#define _LOG_H_

#include<iostream>
#include<ctime>

#define DeBug   0
#define Notice  1
#define Waring  2
#define Error   3

const std::string msg[] = {
    
    
    "DeBug",
    "Notice",
    "Waring",
    "Error"
};

std::ostream &Log(std::string message, int level)
{
    
    
    std::cout << " | " << (unsigned)time(nullptr) << " | " << msg[level] << " | " << message;
    return std::cout;
}
#endif

This is the header file that prints the log;

• mutiServer.cpp

#include "comm.hpp"

int main()
{
    
    
    // 1.创建管道文件
    if (mkfifo(ipcPath.c_str(), MODE) < 0)
    {
    
    
        perror("mkfifo");
        exit(1);
    }

    Log("创建管道文件成功", DeBug) << "step 1" << endl;

    // 2.正常的文件操作
    int fd = open(ipcPath.c_str(), O_RDONLY); // 服务端读取信息
    if (fd < 0)
    {
    
    
        perror("open");
        exit(2);
    }
    Log("打开管道文件成功", DeBug) << "step 2" << endl;

    // 3.编写正常的通信代码
    char buffer[SIZE];
    while (true)
    {
    
    
        memset(buffer, '\0', sizeof(buffer));
        ssize_t s = read(fd, buffer, sizeof(buffer));
        if (s > 0)
        {
    
    
            cout << '[' << getpid() << "] client say: " << buffer << endl;
        }
        else if (s == 0)
        {
    
    
            cerr << '[' << getpid() << "] read end of file, client quit, server quit" << endl;
            break;
        }
        else
        {
    
    
            perror("read");
            break;
        }
    }
    

    // 4.关闭文件
    close(fd); // 关闭管道文件
    Log("关闭管道文件成功", DeBug) << "step 3" << endl;

    unlink(ipcPath.c_str()); // 删除管道文件
    Log("删除管道文件成功", DeBug) << "step 4" << endl;

    return 0;
}

After this code is executed, a named pipe file will be created;

ulink is a system call interface for deleting files;

it will be deleted directly after closing the pipe file;

• client.cpp

#include "comm.hpp"

int main()
{
    
    
    //客户端不需要创建管道了
    //1.打开管道文件
    int fd = open(ipcPath.c_str(), O_WRONLY);
    if(fd < 0)
    {
    
    
        perror("open");
        exit(1);
    }

    //2.通信操作
    string buffer;
    while(true)
    {
    
    
        cout << "Please Enter Command >";
        getline(cin, buffer);
        write(fd, buffer.c_str(), buffer.size());
    }


    //3.关闭管道文件
    close(fd);

    return 0;
}

The client should not create a pipeline file, get it directly, then open it, and write information to the pipeline;

• operation result:
  
  When the server is running, the pipeline file is created, and then the server process is blocked, waiting for the client to write commands;
  
  when the client process is created, both parties open the pipeline file;
  
  the client sends instructions, and the server accepts instructions;
  
  when the client After exiting the process, the server reads 0, also exits the process, closes and deletes the pipeline file;

If multiple child processes are created on the server side to handle client requests:

• mutiServer.cpp

#include "comm.hpp"

void getMessage(int fd)
{
    
    
    char buffer[SIZE];
    while (true)
    {
    
    
        memset(buffer, '\0', sizeof(buffer));
        ssize_t s = read(fd, buffer, sizeof(buffer));
        if (s > 0)
        {
    
    
            cout << '[' << getpid() << "] client say: " << buffer << endl;
        }
        else if (s == 0)
        {
    
    
            cerr << '[' << getpid() << "] read end of file, client quit, server quit" << endl;
            break;
        }
        else
        {
    
    
            perror("read");
            break;
        }
    }

}

int main()
{
    
    
    // 1.创建管道文件
    if (mkfifo(ipcPath.c_str(), MODE) < 0)
    {
    
    
        perror("mkfifo");
        exit(1);
    }

    Log("创建管道文件成功", DeBug) << "step 1" << endl;

    // 2.正常的文件操作
    int fd = open(ipcPath.c_str(), O_RDONLY); // 服务端读取信息
    if (fd < 0)
    {
    
    
        perror("open");
        exit(2);
    }
    Log("打开管道文件成功", DeBug) << "step 2" << endl;

    // 3.编写正常的通信代码
    int pnums = 4;
    for(int i = 0; i < pnums; i++)
    {
    
    
        pid_t id = fork();
        if(id == 0)
        {
    
    
            getMessage(fd);
            exit(1);
        }
    }

    for(int i = 0; i < pnums; i++)
    {
    
    
        pid_t ret = waitpid(-1, nullptr, 0);
    }
    
    // 4.关闭文件
    close(fd); // 关闭管道文件
    Log("关闭管道文件成功", DeBug) << "step 3" << endl;

    unlink(ipcPath.c_str()); // 删除管道文件
    Log("删除管道文件成功", DeBug) << "step 4" << endl;

    return 0;
}

• operation result:
  
  It can be seen that the child processes that execute client tasks are random each time, and multiple processes compete to obtain data;

Note: The difference between .hpp files and .h files
hpp, its essence is to mix the implementation code of .cpp into the .h header file, and the definition and implementation are included in the same file, so the caller of this class only needs to include the hpp file, and there is no need to add cpp to the project to compile. The implementation code will be directly compiled into the obj file of the caller, and no separate obj will be generated. Using hpp will greatly reduce the number of cpp files and compilation times in the calling project, and there is no need to release annoying lib and dll, so it is very Suitable for writing public open source libraries.