Table of contents
Interprocess Communication (IPC)
Interprocess Communication (IPC)
Reference/Citation:
-
How to perform multi-process programming under Linux (preliminary) - Zhihu (zhihu.com) . A brief analysis of several methods of inter-process communication (including example source code) - Zhihu (zhihu.com) .
-
Linux IPC summary (full) xuexingyang's blog-CSDN blog linuxipc .
The original UNIX inter-process communication (IPC: InterProcess Communication) method: including pipes (PIPE), named pipes (FIFO), and signals (Signal); System V process communication methods: including semaphore (Semaphore), message queue (Message Queue) ), and shared memory (Shared Memory). Both of these are early UNIX IPC, as well as sockets and memory mapping. Linux has inherited these eight basic methods.
To sum it up: pipes, named pipes, signals, semaphores, message queues, shared memory, memory maps and sockets.
UNIX IPC
Pipe
A pipe is a half-duplex communication method. It is a buffer in the kernel. Data can only flow in one direction, writing on one end and reading on the other. It can only be used between processes that have a relationship (parent-child process). In addition, the pipe transmits an unformatted byte stream, and the size of the pipe buffer is limited (the pipe buffer exists in memory, and a page size is allocated for the buffer when the pipe is created).
Half-duplex (two-way communication, only one party can send and one can receive at the same time); used for communication between father, son, and brother processes (Can you see the naming method in this patriarchal society? Why don't we rise up and use "parent and child" Process", "peer process" (let's call it!), pipes are also called nameless pipes or anonymous pipes; pipes used for communication between any two processes are called named pipes.
Use the pipe() function to create a buffer for pipe communication. This function will return two file descriptors, namely "read file descriptor" and "write file descriptor", which respectively point to the input end of the buffer. /reading end and output end/writing end, and then two related processes use write() to "write the file descriptor" and write data, and the other uses read() to "read the file descriptor" Just read the data.
#include <unistd.h> int pipe(int pipefd[2]); /* Parameter pipefd array, two file descriptors need to be passed in, pipefd[0] is for reading, pipefd[1] is for writing of*/
When calling the pipe function, a buffer is opened in the kernel for communication. It has a reading end and a writing end. pipefd[0] points to the read end of the pipe, and pipefd[1] points to the write end of the pipe. Therefore, the pipe looks like an open file in the user program. Reading and writing data to this file through read(pipefd[0]) or write(pipefd[1]) is actually reading and writing the kernel buffer.
Operating procedures
The parent process (Author's note: The name here needs to be changed to "parent process", and the "parent node" in the tree structure needs to be changed to "parent node"!!!) calls pipe to open a pipeline and gets two file descriptors pointing to both ends of the pipeline. .
If the parent process (author's note: ahhhh) calls fork to create a child process, then the child process also has two file descriptors pointing to the same pipe.
Take the parent process (author's note: I won't care about it later) as an example of sending data to the child process: the parent process closes the reading end of the pipe, and the child process closes the writing end of the pipe. The parent process can write to the pipe, and the child process can read from the pipe. The pipeline is implemented using a ring queue, and data flows in from the writing end and out from the reading end, thus realizing inter-process communication.
pipe communication example:
The pipe() function returns a file descriptor, so it can only be accessed using the underlying read() and write() system calls.
Pipe read and write rules
When there is no data to read
O_NONBLOCK disable: The read call blocks, that is, the process suspends execution until data arrives.
O_NONBLOCK enable: The read call returns -1, and the errno value is EAGAIN.
when the pipe is full
O_NONBLOCK disable: The write call blocks until a process reads the data.
O_NONBLOCK enable: The call returns -1, and the errno value is EAGAIN.
Four special situations in pipelines
The writing end is closed but the reading end is not closed: then after all the remaining data in the pipe is read, read again will return 0, just like reading to the end of the file.
If the file descriptors corresponding to the write ends of all pipes are closed, read returns 0.
The writing end is not closed, but data is not written, and the reading end is not closed: at this time, after all the remaining data in the pipe has been read, read will be blocked again, and the data will not be re-read and returned until there is data in the pipe to be read.
The read end is closed, but the write end is not closed: At this time, the process will receive the signal SIGPIPE, which usually causes the process to terminate abnormally.
If the file descriptors corresponding to all pipe readers are closed, the write operation will generate the signal SIGPIPE.
The reading end is not closed, but the data is not read, and the writing end is not closed: at this time, when the writing end is full, writing again will block, and data will not be written until there is an empty position in the pipe and return again.
Amount of data written
When the amount of data to be written is no larger than PIPE_BUF (Posix.1 requires PIPE_BUF to be at least 512 bytes), Linux will guarantee the atomicity of the write.
When the amount of data to be written is greater than PIPE_BUF, Linux will no longer guarantee the atomicity of the write.
Disadvantages of using pipes
Two processes can only achieve one-way communication through a pipe. If you want two-way communication, you must create a new pipe or use sockpair to solve this kind of problem.
It can only be used for communication between processes that have affinity relationships, such as parent-child, sibling processes.
Named pipe (FIFO)
Named pipes also provide half-duplex communication, but compared to pipes/unnamed pipes/anonymous pipes, they allow communication between unrelated processes . Named pipes are also called named pipes or real-name pipes (in the Chinese context, as far as I can see, there are now a total of six names for Pipe and FIFO).
Reference: Linux inter-process communication pipe (pipe), named pipe (FIFO) and signal (Signal) - as_ - Blog Park (cnblogs.com), Linux process communication: Named pipe FIFO summary_Mr_John_Liang's blog-CSDN blog .
Detailed reference for named pipe FIFO: [Linux] Inter-process communication-named pipe FIFO programming learning guide blog-CSDN blog named pipe fifo .
FIFO just borrows the file system (named pipe is a special type of file, because everything in Linux is a file, it exists as a file name in the file system) to name the pipe. Processes in write mode write to the FIFO file, while processes in read mode read from the FIFO file. When the FIFO file is deleted, the pipe connection disappears. The advantage of FIFO is that we can identify pipes by the path of the file, allowing connections to be established between unrelated processes.
#include <sys/types.h> #include <sys/stat.h> int mkfifo(const char *filename, mode_t mode); int mknode(const char *filename, mode_t mode | S_IFIFO, (dev_t) 0 ); / * Where pathname is the name of the file to be created (the file must not exist), mode represents the permission bits that will be set on the file and the file type that will be created (indicates its read and write permissions), and dev is the device special file when it is created a value to use. Therefore, it has a value of 0 for first-in-first-out files. */
Operating procedures
-
You can first use access() to determine whether the target named pipe FIFO file exists. If it exists, you can jump to step three, if it doesn't exist, you can jump to step two.
-
Use mkfifo() to create a named pipe FIFO file, using the mode parameter
0777. If the FIFO file created is , you can see the file/tmp/my_fifousing the command line .ls -lF /tmp/my_fifo -
Then use open() (or the advanced package of fopen()) to open the FIFO file (the incoming flags are O_RDONLY, O_WRONLY and O_NONBLOCK, individually or in combination, the incoming flags will be discussed in detail later)
关于 FIFO 读写时候的阻塞问题. Because FIFO is a file, it must be opened before use. -
Then use read/write (or fread/fwrite) to read and write.
-
Finally use close() to close the file.
Reference example
Write the example wirte_fifo.c
#include <stdio.h> #include <unistd.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> #include <string.h> #include <fcntl .h> int main(int argc, const char* argv[]) { if(argc < 2){ printf("./a.out fifoname\n"); exit(1); } // Determine whether the file exists int ret = access(argv[1], F_OK); if(ret == -1) { int r = mkfifo(argv[1], 0777); /* Create a file in the file system that is used Provides FIFO function, that is, named pipes*/ if(r == -1){ perror("mkfifo error"); exit(1); } printf("Named pipe %s was created successfully\n", argv[1]) ; } int fd = open(argv[1],O_WRONLY); if(fd == -1){ perror("open error"); exit(1); }O_WRONLY); char *p = "hello, world"; int len = write(fd, p, strlen(p)+1); close(fd); return 0; }Read example read_fifo.c
#include <stdio.h> #include <unistd.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> #include <string.h> #include <fcntl .h> int main(int argc, const char* argv[]) { if(argc < 2) { printf("./a.out fifoname\n"); exit(1); } // Determine whether the file exists int ret = access(argv[1], F_OK); if(ret == -1) { int r = mkfifo(argv[1], 0664); if(r == -1){ perror("mkfifo error "); exit(1); } printf("The famous pipe %s was created successfully\n", argv[1]); } int fd = open(argv[1],O_RDONLY); if(fd == -1){ perror("open error"); exit(1); } char buf[512]; int len = read(fd, buf, sizeof(buf)); buf[len] = 0; printf("buf = %s\n, len = %d", buf, len); close(fd); return 0; }
Regarding the blocking problem during FIFO reading and writing
Detailed reference:
If open() is used to open a FIFO file in a blocking manner (that is, the O_NONBLOCK flag is not passed in), then read() will be blocked (when the FIFO is empty or other processes are reading, it will be blocked. Until the blocking is released), the same is true for write().
The incoming flags for opening a FIFO file are O_RDONLY, O_WRONLY and O_NONBLOCK, individually or in combination, as detailed below:
A major limitation of opening FIFO is that the program cannot open the FIFO file in O_RDWR mode for read and write operations. The consequences of doing so are not clearly defined. This restriction makes sense because we are using FIFO just to pass data in singletons, so there is no need to use O_RDWR mode. If a pipe opens a FIFO for read/write, the process reads back its own output from the pipe. If you really need to pass data in both directions between programs, it's best to use a pair of FIFOs, one in each direction.
When a Linux process is blocked, it does not consume CPU resources. This process synchronization method is very efficient for the CPU.
Therefore, it can be seen that, in addition to reading/writing, the biggest impact is the O_NONBLOCK flag:
flags = O_RDONLY: open will call blocking and wait until another process opens the same FIFO for writing.
flags = O_WRONLY: open will call blocking and wait until another process opens the same FIFO for reading.
flags = O_RDONLY | O_NONBLOCK: If no other process opens the FIFO for writing at this time, open will also return successfully. At this time, the FIFO will be opened for reading without returning an error.
flags = O_WRONLY | O_NONBLOCK: Return immediately. If no other process is opened for reading at this time, open will fail to open. At this time, the FIFO is not opened and -1 will be returned.
Read and write operations on FIFO files (the impact of passing the O_NONBLOCK flag when open()):
The parameter flag O_NONBLOCK in the open function call will affect the read and write operations of FIFO.
The rules are as follows:
A read call to an empty blocking FIFO will wait until there is data to read before continuing.
A read call to an empty, non-blocking FIFO immediately returns 0 bytes.
A write call to a completely blocked FIFO will wait until data can be written before executing.
Rules regarding the size of data written at a time:
System regulations: If the length of the written data is less than or equal to PIPE_BUF bytes, then either all bytes will be written, or not a single byte will be written. Note the effect of this restriction:
When using only one FIF and allowing multiple different programs to send requests to a FIFO reader process, this restriction is important in order to ensure that data blocks from different programs do not interleave with each other, that is, each operation is atomic. If all write requests are sent to a blocking FIFO, and the data length of each write request is less than or equal to PIPE_BUF bytes, the system can ensure that the data will never be interleaved. It is usually a good idea to limit the length of data passed through the FIFO to PIPE_BUF each time.
In the case of a non-blocking write call, if the FIFO cannot receive all written data, the following rules will be followed:
The length of the data requested to be written is less than PIPE_BUF bytes, the call fails, and the data cannot be written.
The length of the data requested to be written is greater than PIPE_BUF bytes. Partial data will be written and the actual number of bytes written will be returned. The return value may also be 0.
in. PIPE_BUF is the length of the FIFO, which is defined in the header file limits.h. In Linux or other UNIX-like systems, its value is usually 4096 bytes.
Deletion of FIFO files
FIFO files need to be deleted after use to avoid creating junk files.
#include <unistd.h> int unlink(const char *pathname);For details about unlink, refer to: unlink(2) - Linux man page .
Signal
Additional citations/references: Chapter 10 Signal - as_ - Blog Garden (cnblogs.com) , Linux-Application Programming-Learning Summary (4): Inter-process Communication (Part 2)_Studying Broccoli’s Blog-CSDN Blog , Linux Inter-Process Communication pipes (pipe), named pipes (FIFO) and signals (Signal) - as_ - Blog Park (cnblogs.com) .
Signals are used to notify the receiving process that an event has occurred. A signal is delivered for asynchronous notification (signals are asynchronous, and a process does not have to wait for the arrival of the signal through any operations. In fact, the process does not know when the signal arrives. Signal It is the only asynchronous communication mechanism among inter-process communication mechanisms and can be regarded as an asynchronous notification) and cannot transmit any data. Signals are a kind of interrupt mechanism at the software level, and the simulation effect is similar to the interrupt mechanism.
However, signals and interrupts are different. The response and processing of interrupts occur in kernel space, while the response of signals occurs in kernel space, and the execution of signal handlers occurs in user space.
So, when do you detect and respond to a signal? Usually happens in two situations:
After the current process enters the kernel space due to a system call, interrupt or exception, and before returning from the kernel space to the user space, that is, when a process is about to return from the kernel state to the user state (that is, the soft interrupt signal does not work in the kernel state) , it will not be processed until returning to user mode).
When the current process is just woken up after entering sleep in the kernel, it returns to user space early due to the detection of the signal, that is, when a process is about to enter or leave an appropriate low-priority sleep state.
Reference signal (LINUX signal mechanism)_Baidu Encyclopedia (baidu.com) .
In computer science, signals are a restricted method of inter-process communication in Unix, Unix-like, and other POSIX-compliant operating systems. It is an asynchronous notification mechanism used to remind the process that an event has occurred. When a signal is sent to a process, the operating system interrupts the normal flow of control of the process. At this time, any non-atomic operations will be interrupted. If the process defines a signal handler, it will be executed, otherwise the default handler will be executed.
Processes can send soft interrupt signals to each other through the system call kill (kill command is used in shell, kill() function is used in application programming). The kernel can also send signals to the process due to internal events, notifying the process that an event has occurred. Note that signals are only used to notify a process of what events have occurred and do not pass any data to the process. The shell can also use signals to pass job control commands to its child processes.
Signal type:
Common signals:
signal name signal number processing content SIGINT 2 When Ctrl+C, the OS sends a message to each process in the foreground process group. FOLLOWS 3 When entering the Quit Key (CTRL+/), it is sent to all Foreground Group processes. SIGABRT 6 The abort function is called and the process terminates abnormally. SIGKILL 9 Abort a process. Cannot be ignored and captured. TARGET TERM 15 Requests to terminate the process. The kill command is sent by default. The OS default termination signal sent by the kill command SIGTSTP 20 Suspend Key, usually Ctrl+Z. Sent to all Foreground Group processes SIGSTOP 19 Abort the process. Cannot be ignored and captured. SIGCONT 18 When the stopped process resumes operation, it is automatically sent. SIGSEGV 11 The OS issues this signal when an invalid storage access occurs. SIGPIPE 13 Involves pipes and sockets. Sent when writing to the Pipe after the reader has terminated. Occurs when writing to a pipe that no one has read. SIGALRM 14 The timer timeout set by the alarm function or the interval timer timeout set by the setitimer function SIGCHLD 17 The OS sends this signal to its parent process when the child process terminates or stops. When the process Terminate or Stop, SIGCHLD will be sent to its parent process. By default, this Signal will be ignored SIGPOLL / SIGIO 8 Indicates an asynchronous IO event, mentioned in Advanced IO SIGUSR1 10 User-defined signals, whose functions and meanings are defined by the application itself SIGUSR2 12 User-defined signals, whose functions and meanings are defined by the application itself SIGTTIN 21 Background process wants to read SIGTTOU 22 The background process wants to write View all signals and corresponding numbers in the Shell:
kill -l.A signal that cannot be ignored
SIGKILL, it will end the process.
SIGSTOP, which is part of the job control mechanism, will suspend the execution of the job. Cannot be caught or ignored.
This is to provide a sure way to Stop or Kill a Process.
job control signal
SIGCHILD: The child process has been stopped or terminated;
SIGCONT: If the process stops, let it continue running;
SIGSTOP: stop signal (cannot be caught or ignored);
SIGTSTP: Stop signal for interaction;
SIGTTIN: Members of the background process group read the control terminal;
SIGTTOU: Members of the background process group write to the controlling terminal.
Process job control in Shell: linux task control bg, fg, jobs, kill, wait, suspend... - Baidu Experience (baidu.com) . There are bg, fg, jobs, kill, wait and suspend.
Signal transmission:
-
Between drivers and applications: Signals can directly interact between user space processes and kernel processes (such as drivers). The kernel process can use it to notify user space processes of what events have occurred (for example, drivers commonly use SIGIO signals to asynchronously notify applications ).
-
Between applications: It can also be used as a way of inter-process communication or behavior modification, explicitly sent by one process to another process (or itself). The generation of a signal is called generation, and the reception of a signal is called capture.
Situations that generate signals:
-
Issued by Shell users. For example, pressing CTRL+C on a process in the foreground generates a SIGINT signal and sends it to the foreground process.
-
System call API in user mode process (such as kill(), raise(), alarm(), pause(), etc.).
-
Drivers send signals to notify applications asynchronously (common ones such as SIGIO), or hardware errors.
Three types of processing for captured signals:
-
The process ignores the signal. Most signals can be ignored. Furthermore, if we ignore the signals generated by some hardware exceptions, the behavior of the process is undefined. In fact, the default action for individual signals is to ignore them.
-
Capture the signal. After the process receives the signal, it executes the function set by the user to call the signal / sigaction system (the user can set the signal callback function).
-
Perform the default action. If no processing is performed, the default action is performed. The default behavior of most signals is to abort the process.
The default behavior of some signals not only terminates the process, but also generates a core dump, that is, a file named core is generated, which saves the image of the process memory at exit, which can be used for debugging. In the following situations, the core file will not be generated:
The current process does not belong to the current user.
The current process does not belong to the current group.
The user does not have write permission in the current directory.
The Core file already exists and the user does not have write permission.
The file is too large and exceeds RLIMIT_CORE.
The essence is: the signal asynchronously notifies the process that receives the signal that an event has occurred, and then the operating system will interrupt the execution of the process that received the signal and instead execute the corresponding signal handler (execute according to ignore, capture or default operations) ).
#include <sys/types.h>
#include <signal.h>
#include <unistd.h>
void
(*signal(int sig,void (*func)(int)))(int);
/* binding
The first parameter of the callback function after receiving a certain signal is the signal, and the second parameter is the user's own processing function pointer for the signal.
The return value is a pointer to the previous signal handler.
Example
: int ret = signal(SIGSTOP, sig_handle);
*/
/
* Since signal is not robust enough, it is recommended to use the sigaction function. The sigaction function reimplements the signal function*/
int sigaction(int signum, const struct sigaction *act,struct sigaction *oldact);
/* Regarding the use of sigaction, check the usage time */
//
The kill function sends a signal to the process with the process number pid, and the signal value is sig. When pid is 0, the signal sig is sent to all processes in the current system.
int kill(pid_t pid,int sig);
/*
The pid parameter of kill has four situations:
1).pid > 0, the signal is sent to the process with the process ID of pid;
2).pid == 0, the signal is sent to all processes that belong to the same process group as the sending process (the process group ID of these processes is equal to the process group ID of the sending process), and the sending process has the permission to send signals to these processes. Note that the term "all processes" does not include the implementation-defined set of system processes. For most UNIX systems, this system process set includes the kernel process and init (pid 1);
3).pid < 0, the signal is sent to all those whose ID is equal to the absolute value of pid, and the sender has permission to send signals process. As above, the all process set does not include system processes.
4).pid == -1, sends this signal to all processes on the system to which the sending process has permission to send preferences. As before, no specific system processes are included.
Example
: End the parent process kill(getppid(), SIGKILL);
*/
//
Bootstrap a signal sig to the current process, that is, send a signal to the current process. Equivalent to kill(getpid(),sig);
int raise(int sig);
//
alarm() is used to set the signal SIGALRM to be sent to the current process after the number of seconds specified by the parameter seconds. If the seconds parameter is 0, the previously set alarm will be canceled and the remaining time will be returned. When using the alarm function, pay attention to the coverage of the alarm function. That is, if the alarm function is used once in a process, the previous alarm function of the process will be invalid.
//After seconds, send the SIGALRM signal to the process itself.
unsigned int alarm(unsigned int seconds);
/*
When the set time value is exceeded, the SIGALRM signal is generated. If this signal is not ignored or caught, the action is to terminate the process.
If the alarm time has been previously set for the process when alarm is called, and it has not timed out, the remaining value of the alarm time is returned as the value of this alarm function call. The previously registered alarm time is replaced by the new value. If there is a previously registered alarm time that has not yet passed, and the seconds value is 0, the previous alarm time is cancelled, and the remaining values are still used as the return value of the function.
*/
//
Delay/sleep seconds seconds
unsigned int sleep(unsigned int seconds);
/*
Return 0 or the number of seconds without sleep.
This function causes the calling process to be suspended until:
(1) the wall clock time specified by seconds has passed;
(2) the process captures a signal and returns from the signal handler.
Like the alarm signal, due to some system activity, the actual return time will be later than required
*/
//
Make the calling process (or thread) sleep until the signal is received, or terminate, or cause it to call a signal capture function.
// The pause function causes the calling process to suspend until a signal is caught.
int pause(void);
/*
Pause returns only when a signal handler is executed and returned from it. In this case, pause returns -1 and errno is set to EINTR.
*
/
//The function of the abort function is to cause the program to terminate abnormally. This function sends the SIGABRT signal to the calling process. Processes should not ignore this signal.
// The abort function never returns.
void abort(void);
Some examples of sending and catching signals:
-
Linux system calls wait(), signal(), kill() | feng yanfengyu (feng-qi.github.io) .
-
Linux's SIGUSR1 and SIGUSR2 signals - Xiaolou listened to the spring rain all night - Blog Park (cnblogs.com) . Linux's SIGUSR1 and SIGUSR2 signals_diaoqu4574's blog-CSDN blog .
Use signal() to register the callback function for the SIGUSR1 signal, and then use in the shell
kill -USR1 <进程号>to send the SIGUSR1 signal.More commonly used signals: Use the kill command to send signals. When specifying the signal name, remove the "SIG" prefix. For example, if you want to send SIGUSR1, specify it
-USR1.INT 2 interrupt (same as Ctrl + C)
QUIT 3 exit (same as Ctrl + \)
TERM 15 Termination
KILL 9 Forced termination
CONT 18 Continue (opposite of STOP, fg/bg command)
STOP 19 Pause (same as Ctrl + Z)
-
More can be found online.
System V IPC
System V IPC refers to the three inter-process communication tools introduced by AT&T in the System V.2 release: (1) semaphores, used to manage access to shared resources (2) shared memory, used to efficiently implement processes Data sharing between processes (3) Message queue, used to realize the transfer of data between processes. We refer to these three tools collectively as System V IPC objects, and each object has a unique IPC identifier. To ensure that different processes can obtain the same IPC object, an IPC key must be provided, and the kernel is responsible for converting the IPC key into an IPC identifier.
System V IPC has similar syntax, in terms of API naming: semxxx() for semaphores, shmxxx() for shared memory, msgxxx() for message queues.
System V IPC generally operates as follows:
1. Select the IPC keyword (formal parameter key_t key in API). You can use the following three methods:
IPC_PRIVATE. The kernel is responsible for selecting a keyword and then generating an IPC object and passing the IPC identifier directly to another process.
Simply select an existing keyword.
Use the ftok() function to generate a keyword.
2. Use the semget()/shmget()/msgget() function to create or access an IPC object based on the IPC keyword key and a flag.
If the key is IPC_PRIVATE, or the key is not associated with an existing IPC object and the flag contains the IPC_CREAT flag, a new IPC object will be created.
3. Use the semctl()/shmctl()/msgctl() function to modify the properties of the IPC object.
4. Use the semctl()/shmctl()/msgctl() function and the IPC_RMID flag to destroy the IPC object.
System V IPC sets an ipc_perm structure for each IPC object and initializes it when creating the IPC object. This structure defines the access permissions and owners of the IPC object:
struct ipc_perm{ uid_t uid; //Owner's user id gid_t gid; //Owner's group id uid_t cuid; //Creator's user id gid_t cgid; //Creator's group id mode_t mode; //Access mode … };Message queues, semaphores, and shared memory are collectively called XSI IPC. They have similar IPC structures in the kernel (msgid_ds for message queues, semid_ds for semaphores, and shmid_ds for shared memory), and are all identified by a non-negative integer. The identifier is referenced (msg_id of the message queue, sem_id of the semaphore, and shm_id of the shared memory, obtained through msgget, semget, and shmget respectively). The identifier is the internal name of the IPC object. Each IPC object has a key (key_t key) Association, using this key as the external name of the object.
The IPC structure of XSI IPC works system-wide and does not use reference counting. If a process creates a message queue and puts several messages in the message queue, after the process terminates, the message queue and its contents remain even if no program is using the message queue. When PIPE terminates the last process that referenced the pipe, the pipe is completely deleted. When the last process that references the FIFO terminates, although the FIFO is still in the system, its contents will be deleted. Unlike PIPE and FIFO, XSI IPC does not use file descriptors, so you cannot use ls to view IPC objects, you cannot use the rm command to delete them, and you cannot use the chmod command to delete their access rights. You can only use ipcs and ipcrm to see if you can delete them.
The commands for managing IPC objects in the shell are ipcs, ipcmk and ipcrm.
like:
ipcs -sCheck the number of created semaphore sets andipcrm -s <semid>delete a semaphore set numbered semid.
ipcs -mCheck the number of shared memories created andipcrm -m shm_iddelete the shared memory.
Semaphore
Additional reference/citation: IPC object semaphore's blog - CSDN blog ipc semaphore , IPC semaphore detailed explanation Qiuoooooo's blog - CSDN blog ipc semaphore .
A semaphore is a counter, which is used to record the access status of each process to a certain resource, and can be used to control the access of multiple processes to shared resources (for example, semaphores are used in the later shared memory). It is often used as a locking mechanism to prevent other processes from accessing a shared resource when a process is accessing the resource. It is often used to deal with the access synchronization problem of critical resources (critical resources: resources that can only be operated by one process or thread at a certain time). Only one thread can access critical resources at any time.
Semaphore workflow:
(1) Create a semaphore to control a resource.
(2) If the value of this semaphore is positive, the resource is allowed to be used. The process decrements the advance number by 1.
(3) If the semaphore is 0, the resource is currently unavailable and the process enters the sleep state until the semaphore value is greater than 0, the process is awakened and goes to step (1).
(4) When the process no longer uses a resource controlled by a semaphore, the semaphore value is increased by 1. If there is a process sleeping waiting for this semaphore at this time, wake up this process.
When a process no longer uses the resource, the semaphore is +1 (the corresponding operation is called the V operation). On the contrary, when a process uses the resource, the semaphore is -1 (the corresponding operation is the P operation). All value operations on semaphores are atomic operations.
P operation, prepare to start reading and writing, P(sv): If the value of sv is greater than zero, decrement it by 1; if its value is zero, suspend the execution of the process and wait for the operation.
V operation can be released after reading and writing. V(sv): If other processes are suspended waiting for sv, let it resume running. If no process is suspended waiting for sv, add 1 to it.
A simple understanding is that P is equivalent to applying for resources, and V is equivalent to releasing resources.
For example, two processes share the semaphore sv, with an initial value of 1. Once one of the processes performs the P(sv) operation, it will get the semaphore and can enter the critical section, reducing sv by 1. And the second process will be blocked from entering the critical section because when it tries to execute P(sv), sv is 0, it will be hung waiting for the first process to leave the critical section and execute V(sv) to release the semaphore , then the second process can resume execution.
Binary semaphore : value is 0 or 1. Similar to a mutex lock, the value is 1 when the resource is available and 0 when it is unavailable; that is, the P operation is equivalent to locking, and the V operation is equivalent to unlocking.
Counting semaphore : value between 0 and n. Same as statistics resources, its value represents the number of available resources.
In a Linux system, using a semaphore usually requires four operations: creating a semaphore, initializing the semaphore, semaphore PV operation, and semaphore deletion.
Create/get a semaphore collection:
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
int semget(key_t key, int nsems, int semflg);
/*
key: semaphore collection number/key The value
can be obtained by using the function key_t ftok(const char *pathname, int proj_id);
different processes can access the same semaphore set as long as the key value is the same.
There is a special value IPC_PRIVATE, which means creating a private semaphore
nsems for the current process: this parameter Indicates the number of semaphores in the semaphore set you want to create. Semaphores can only be created as collections.
The number of semaphores to be created, usually 1. If multiple semaphores are created, it is called a semaphore set.
If you are creating a new collection, nsems must be specified.
If referencing an existing semaphore collection, specify nsems as 0. If you are just getting the identifier of the semaphore set (rather than creating a new one), nsems can be 0.
semflg:
IPC_CREAT|IPC_EXCL means that if the semaphore corresponding to the key does not exist, it will be created, and if it does, an error will be reported. That is, a new semaphore collection will be created, or -1 will be returned if it already exists.
IPC_CREAT means that if the semaphore corresponding to the key does not exist, it will be created, and if it exists, the identifier of the semaphore will be returned directly. Returns a new or existing semaphore collection.
The lower 8 bits of the flag are used as the access permission bits of the semaphore, similar to the access permissions of the file.
The lower 8 bits of the flag are the permission bits. Generally, 0666 is used (the binary number of 6 is 110, which means readable, writable, and non-executable. The three 6s correspond to the current user, group user, and other users respectively). For example, flag can be IPC_CREAT|0666. The return value is: the semaphore is returned
successfully
. Semid of the collection (non-negative integer), returns -1 on failure.
For example, if the same semaphore is used between process A and process B, then design A generates/creates the semaphore first, and then B references/binds the semaphore: Therefore, when A calls
semget(), the semflg parameter should be passed in IPC_CREAT|IPC_EXCL |0666, and nsems is the number of semaphores to be created;
and B should pass in IPC_CREAT|0666 or IPC_CREAT, and nsems is 0.
*/
key_t
ftok( const char * fname, int id);
/* Format conversion function of IPC key value. The system must specify an ID value when establishing IPC communication (message queue, semaphore and shared memory). Normally, the id value is obtained through the ftok function.
fname is the file name you specify (an existing file name). Generally, the current directory
id is the sub-serial number. Although it is an int type, only 8bits (1-255) are used.
Calculation process: If the index node number of the specified file is 65538, converted to hexadecimal is 0x010002, and the ID value you specified is 38, converted to hexadecimal is 0x26, then the final key_t return value is 0x26010002.
It is used to ensure that the same program and two identical programs under two different users obtain IPC key values that do not interfere with each other.
Example key_t key = ftok(".", 'a');
*/
Setting the semaphore (initialization value or destruction):
/* Structure defined in the kernel*/
union semun{
int val; // The value used by SETVAL
struct semid_ds *buf; // The buffer area used by IPC_STAT and IPC_SET
unsigned short *array; // The buffer area used by GETALL and SETALL, ALL, all semaphores of a certain semaphore set
struct seminfo *__buf; // IPC_INFO (linux-specific) uses the cache area
};
/
* The kernel maintains a semid_ds structure for each semaphore set */
struct semid_ds{
struct ipc_perm sem_perm;
unsigned short sem_nsems;
time_t sem_otime;
time_t sem_ctime;
...
}
int
semctl(int semid, int semnum, int cmd, union semun arg);
/*
semid: fill in the number cmd of the semaphore set to be operated
: To operate the semnum-th semaphore in the semaphore set semid (range: 0 ~ nsems-1)
cmd: A variety of different operations are defined in the header file sem.h: The following example is
IPC_STAT: Get the semid_ds structure of a certain semaphore collection and store it to the address pointed to by the buf parameter of the semun union.
IPC_SET: Set the value of the ipc_perm member of the semid_ds structure of a certain semaphore set. The value is taken from the buf parameter of the semun union.
IPC_RMID: The kernel deletes the semaphore collection.
GETVAL: Returns the value of a semaphore in the collection.
SETVAL: Sets the value of a single semaphore in the collection to the value of the val member of the union.
The two commonly used values of cmd are:
SETVAL: initialize the value of the semnum-th semaphore to arg.val;
IPC_RMID: delete the semaphore.
Generally, it means setting the initial value and deleting the semaphore. The above cmd defines many different operations. You can check it when you use it (search online or use man to view the kernel manual). Return value: Success: IPC_STAT, IPC_SETVAL or IPC_RMID operation: 0
, IPC_GETVAL operation: returns the value of the current semaphore; failure: returns -1.
*/
Semaphore operation (P/V operation, changing the value of the semaphore):
/* A structure maintained in the kernel, used to describe what kind of operation is performed on a certain semaphore */
struct sembuf
{
unsigned short sem_num; /* semaphore number */
short sem_op; /* semaphore operation */
short sem_flg; / * operation flags */
}
/*
sem_num: Which semaphore in the semaphore set, starting from 0
sem_op: Indicates the operation of the semaphore (P operation or V operation). If its value is positive, the value is added to the existing signal content. Usually used to release the right to use the controlled resource; if the value of sem_op is a negative number and its absolute value is greater than the current value of the signal, the operation will block until the signal value is greater than or equal to the absolute value of sem_op. Usually used to obtain the right to use resources.
The value -1 is P operation, and the value 1 is V operation.
sem_flg: signal operation flag, which has two values: IPC_NOWAIT and SEM_UNDO: IPC_NOWAIT
: When the operation of the semaphore cannot be satisfied, semop() will not block, but will return immediately and set the error message;
SEM_UNDO: program When it ends (whether it is normal or abnormal), it is guaranteed that the signal value will be set; at the same time, if the resource is not released when the process ends, the system will automatically release it. For example, the initial value of the semaphore is 20, and process a operates the semaphore in SEM_UNDO mode
. The count value of the semaphore,
For example, the initial value of the semaphore is 20. Process a operates the semaphore in SEM_UNDO mode and increments it by 1; when the process does not exit, the semaphore becomes 21; when the process exits, the value of the semaphore returns to 20 */int semop(
int
semid
, struct sembuf *sops, unsigned nsops);
int semtimedop(int semid, struct sembuf *sops, unsigned nsops,struct timespec *timeout);
/*
semid: semaphore collection number
sops: the operation to be performed, first fill in the struct sembuf structure body, and then pass in its address, you can pass in an array (formal parameter struct sembuf sops[])
nsops: indicates the number of semaphores to be operated. The sops parameter can pass an array, and nsops represents the number of sops. One sops corresponds to the operation of a semaphore,
so multiple semaphores in a collection can be operated at the same time.
Return value: 0 is returned on success, -1 on failure.
*/
/
* Example, operate on a semaphore V, that is, semaphore +1 */
struct sembuf sops_v = {0, +1, SEM_UNDO}; // Add one semop to the semaphore with index value 0
(semid, &sops, 1); // The above function is executed once
manual:
Process 1 (sem):
①Call semget to create a semaphore;
②Call SETVAL of semctl to set an initial value for the semaphore;
③Call semop to perform P and V operations
Process 2 (sem2):
①Call semget to obtain the identifier semid of the existing semaphore;
②Call semop to perform P operation and V operation.
(Note: If another process also uses the semaphore and performs a P operation to make the value of the semaphore -1, then when this process performs the P operation, it will block and wait until another process enters the city to perform the V operation +1 release resources)
Example: Refer to the example given in Qiuoooooo's blog - CSDN blog ipc semaphore , a detailed explanation of IPC semaphores .
Message Queue
A message queue is a linked list of messages, stored in the kernel and identified by a message queue identifier. Message queues overcome the shortcomings of less signal transmission information, pipes can only carry unformatted byte streams, and limited buffer sizes. The message list is stored in the kernel, and each message queue is identified by a message queue identifier; unlike pipes, message queues are stored in the kernel, and a message queue can only be deleted when the kernel is restarted; the size of the message queue is limited.
Message queues have a specific format and a specific priority. A process with write permissions can add new messages to the message queue; a process with read permissions can read messages from the message queue.
Message queue is used for inter-process communication running on the same machine. It is very similar to a pipe. In fact, it is a communication method that is gradually being eliminated. We can replace it with a stream pipe or socket .
I won’t mention the usage here, just check it when you use it.
Shared Memory
Arguably the most useful method of inter-process communication and the fastest form of IPC. Shared memory is the fastest IPC method and is specially designed for the inefficiency of other inter-process communication methods.
Usually one process creates a shared memory area, and other processes read and write to this memory area. Shared memory is to map a section of memory that can be accessed by other processes. This shared memory is created by one process, but can be accessed by multiple processes (as long as the page tables of the two processes are modified so that their virtual addresses map to the same physical page can be shared). The Proc A process writes data to the memory, and the Proc B process reads the data from the memory. During this period, a total of two copies occur: Proc A to shared memory and shared memory to Proc B. Because it operates directly on the memory, it is shared. The speed of memory is also increased. The common way is to use the shmXXX function family to use shared memory for storage.
When two processes map virtual addresses to physical addresses through page tables, there is a common memory area in the physical address, that is, shared memory. This memory can be seen by both processes at the same time. In this way, when one process performs a write operation and another process performs a read operation, inter-process communication can be achieved.
However, because shared memory does not provide a corresponding mutual exclusion mechanism, shared memory is generally used in conjunction with semaphores, which requires programmer control. Shared memory is often used in conjunction with other communication mechanisms, such as semaphores, to achieve synchronization and communication between processes. We want to ensure that a process cannot be read while writing, so we use semaphores to achieve synchronization and mutual exclusion.
Create shared memory:
#include <sys/types.h> #include <sys/ipc.h> #include <sys/shm.h> int shmget(key_t key, size_t size, int shmflg); /* key: and the semaphore introduced above The parameter key of the semget function is the same, which is mainly used to distinguish processes. You can use the function key_t ftok(const char *pathname, int proj_id); to obtain size: indicating the size of the shared memory to be applied for, which is generally an integer multiple of 4k (i.e. 4096xn bytes). flags: IPC_CREAT and IPC_EXCL are used together to create a new shared memory, otherwise -1 is returned. for creating new ones. IPC_CREAT returns a shared memory when used alone. If it exists, it will be returned directly. If not, it will be created. Used to reference/bind an existing key. Same as semflag of semget(). For example: IPC_CREAT|IPC_EXCL|0666 Return value: The ID of the shared memory, that is, shmid, is returned successfully, and -1 is returned on failure. */
Set shared memory properties:
/* shmid_ds structure, sends the parameters of the setting command to the shared memory */
strcut shmid_ds{
struct ipc_perm shm_perm;
size_t shm_segsz;
time_t shm_atime;
time_t shm_dtime;
...
}
int
shmctl(int shmid,int cmd,const void* addr);
/*
Commonly used values of cmd are:
(1) IPC_STAT gets the shmid_ds structure of the current shared memory and saves it in buf
(2) IPC_SET uses the value in buf to set the shmid_ds structure of the current shared memory
(3) IPC_RMID Delete the current shared memory.
When cmd is IPC_RMID, it can be used to delete a piece of shared memory, for example: shmctl(shm_id, IPC_RMID, NULL);
*/
The key point to note when using shared storage to implement inter-process communication is the synchronization of data access. You must ensure that when a process reads data, the data it wants has already been written. Usually, semaphores are used to synchronize access to shared storage data. In addition, this can be achieved by using the shmctl function to set certain flag bits of the shared storage memory such as SHM_LOCK, SHM_UNLOCK, etc.
Hook/unhook functions:
void *shmat(int shm_id,const void *shmaddr,int shmflg); /* The shmat function connects the shared memory to the address space of the process through shm_id and mounts the applied shared memory on the page table of the process. It is to connect the virtual Memory corresponds to physical memory. shmid: Fill in the id of the shared memory. The shmaddr parameter can be specified by the user to map the address of the shared memory to the process space. If shm_addr is 0, the kernel tries to find an unmapped area. shmaddr is usually NULL, and the system selects the address of the shared memory to be attached. shmflg can be returned for SHM_RDONLY Value: Returns the virtual address of this memory, which the process can read and write. Example usage: char* mem = (char*)shmat(shm_id, NULL, 0); */ int shmdt(const void *shmaddr); /* The function of shmdt is to detach the shared memory from the page table and remove the mapping relationship between the two. shmaddr: represents the virtual address of this physical memory. Return value: -1 on failure. Example usage: shmdt(mem); */
manual:
-
Use ftok() to obtain the key_t key.
-
Use shmget() to create/reference a piece of shared memory. Then use shmat() to return the address of the shared memory corresponding to the shmid number.
-
Just read and write the address of the shared memory directly.
-
When it is no longer in use, use shmdt() to cancel the link; then use shmctl() to destroy the shared memory.
For mutually exclusive access to shared memory, you can use a semaphore or lock mechanism. A method of locking mutually exclusive access to shared memory for inter-process communication: linux | How to lock inter-process communication - Zhihu (zhihu.com ) .
IPC extra mode
Memory Map
That is, map a file to a piece of memory through the mmap() function, and then perform read and write operations. After both processes map the same file, they can read and write separately without calling the I/O API but directly operating the memory.
Each process using this mechanism achieves communication between multiple processes by mapping the same shared file to its own process address space (this is similar to shared memory, as long as one process operates on the memory of the mapped file, Other processes can also be seen immediately).
#include <sys/mman.h> #include <unistd.h> void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset); int munmap(void *start, size_t length);
void *mmap(void*start,size_t length,int prot,int flags,int fd,off_t offset); //Themmap function maps a file or other object into memory. The first parameter is the starting address of the mapping area. Setting it to 0 means that the system determines the starting address of the mapping area. The second parameter is the length of the mapping. The third parameter is the expected memory protection flag. The fourth parameter is Specifies the type of mapping object, the fifth parameter is the file descriptor (indicates the file to be mapped), and the sixth parameter is the starting point of the content of the mapped object. The pointer to the mapped area is returned successfully, and MAP_FAILED [its value is (void *)-1] is returned on failure. int munmap(void* start,size_t length); //The munmap function is used to cancel the starting address of the mapped memory pointed to by the parameter start, and the parameter length is the memory size to be cancelled. If the unmapping is successful, 0 is returned, otherwise -1 is returned, and the cause of the error is stored in the error code EINVAL in errno. int msync(void *addr,size_t len,int flags); //msync function realizes the consistency of disk file content and shared memory access content, that is, synchronization. The first parameter is the address of the file mapped to the process space, the second parameter is the size of the mapped space, and the third parameter is the refresh parameter setting.
Detailed explanation of the API can be found in the document: 【Linux 应用开发】\0-用到的API-收集积累\文件IO、字符流收发和字符串处理相关的API收集积累.c. More can be found online.
mmap() Detailed explanation of Linux memory management - Detailed explanation of mmap principle - Zhihu (zhihu.com) .
Example: The example here is to map a file to a piece of memory through mmap(), and then read this memory. There is another process that maps the same file through mmap() and can be modified.
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <string.h>
#include <sys/mman.h>
#include <fcntl.h>
int main(int argc, const char* argv[])
{
int fd = open("english.txt", O_RDWR);
if(fd == -1){ perror("open error"); exit(1); }
// get file length
int len = lseek(fd, 0, SEEK_END);
void * ptr = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd,0);
printf("buf = %s\n", (char *)ptr); //Read data from memory
char
buf[4096];
close(fd);
if(ptr == MAP_FAILED){ perror("mmap error"); exit(1); }
data
// Release the memory mapping area
int ret = munmap(ptr, len);
if(ret == -1){ perror("munmap error"); exit(1); }
return 0;
}
The difference between shared memory and memory mapped files :
Memory mapped files use virtual memory to map files into the address space of the process. After that, the process operates the file just like it operates the address in the process space, such as using memory operation functions such as memcpy in the C language. This method can be well applied in situations where a file or a large file needs to be processed frequently. This method of processing IO is more efficient than ordinary IO.
Shared memory is a special case of memory-mapped files, which map a block of memory rather than a file on disk. The subject of shared memory is process. The operating system will allocate a memory space to each process by default. Each process is only allowed to access the memory allocated to it by the operating system, and cannot access other processes. Sometimes we need to access the same memory between different processes, what should we do? The operating system provides APIs for creating and accessing shared memory. Processes that need to share memory can use this set of defined APIs to access the memory shared between multiple processes. Accessing this memory by each process is like accessing a memory on a hard disk. Files are the same.
The difference and connection between memory mapped files and virtual memory :
Memory mapped files and virtual memory are both important parts of operating system memory management. They have similarities and differences.
Connection: Virtual memory and memory mapping are both mechanisms for loading part of the content into memory and placing another part on disk. All are transparent to users.
Difference: Virtual memory is a part of the hard disk and is the data exchange area between the memory and the hard disk. During the running of many programs, temporarily unused program data is put into this virtual memory to save memory resources. Memory mapping is a mapping of a file to a block of memory, so that the program can access the file through the memory pointer.
The hardware foundation of virtual memory is the paging mechanism. Another basis is the principle of locality (temporal locality and spatial locality), so that part of the program can be loaded into the memory, and the rest remains in the external memory. When the accessed information does not exist, the required data can be transferred into the memory. The memory mapped file is not local, but allows all or part of the contents of the silver snake disk in a certain area of the virtual address space to access the mapped disk file through this area without the need for file I/O and no need to The file contents are buffered.
Socket
When creating a Socket, select the scope for use within the system (and select Ethernet for TCP/UDP/IP communication between different machines).
A socket has three attributes: domain, type, and protocol, corresponding to different domains. The socket also has an address as its name.
The domain specifies the protocol family used for socket communication. The most commonly used domain is AF_INET, which represents network sockets. The underlying protocol is the IP protocol. For network sockets, since the server may provide multiple services, the client needs to use the IP port number to specify a specific service. AF_UNIX stands for local socket, implemented using Unix/Linux file system.
The IP protocol provides two communication methods: streams and datagrams (corresponding to the TCP protocol and UDP protocol respectively). The corresponding socket types are stream sockets and datagram sockets respectively. Character. Streaming sockets (SOCK_STREAM) are used to provide connection-oriented, reliable data transmission services. This service ensures that data is sent error-free, duplicate-free, and received in order. Streaming sockets use the TCP protocol. Datagram sockets (SOCK_DGRAM) provide a connectionless service. This service does not guarantee the reliability of data transmission. Data may be lost or duplicated during transmission, and data cannot be guaranteed to be received in sequence. Datagram sockets use the UDP protocol.
#include <sys/types.h> #include <sys/socket.h> int socket(it domain,int type,int protocal); int bind(int socket,const struct sockaddr *address,size_t address_len); int listen(int socket,int backlog); int accept(int socket,struct sockaddr *address,size_t *address_len); int connect(int socket,const struct sockaddr *addrsss,size_t address_len); ssize_t send(int sockfd, const void *buf, size_t len, int flags); ssize_t recv(int sockfd, void *buf, size_t len, int flags); ssize_t recvfrom(int sockfd, void *buf, size_t len, int flags, struct sockaddr *src_addr, socklen_t *addrlen); ssize_t sendto(int sockfd, const void *buf, size_t len, int flags, const struct sockaddr *dest_addr, socklen_t addrlen);
Refer to 【Linux 应用开发】\3-Socket编程\the content inside. More can be found online.