Condensed Notes on Operating Systems (4)-Process Management

 Main quotes from the article notes:

Axiu’s study notes (interviewguide.cn)

Kobayashi coding (xiaolincoding.com)

The differences and connections between processes, threads and coroutines

	process	thread	coroutine
definition	The basic unit of resource allocation and ownership	The basic unit of program execution	A lightweight thread in user mode, the basic unit of thread internal scheduling
switching situation	Saving of the process CPU environment (stack, registers, page tables, file handles, etc.) and setting of the newly scheduled process CPU environment	Save and set the contents of the program counter, a few registers, and the stack	Save the register context and stack first, and then restore them when switching back.
switcher	operating system	operating system	user
switching process	User mode->Kernel mode->User mode	User mode->Kernel mode->User mode	User mode (not trapped in the kernel)
call stack	kernel stack	kernel stack	user stack
Have resources	CPU resources, memory resources, file resources and handles, etc.	Program counter, registers, stack and status word	Has its own register context and stack
Concurrency	Switch between different processes to achieve concurrency, each occupying the CPU to achieve parallelism	Multiple threads within a process execute concurrently	Only one coroutine can be executed at the same time, while other coroutines are in a dormant state, which is suitable for time-sharing processing of tasks.
System overhead	Switching virtual address space, switching kernel stack and hardware context, CPU cache invalidation, page table switching, very expensive	Only a few register contents need to be saved and set when switching, so the overhead is small	Directly operating the stack basically has no kernel switching overhead, and global variables can be accessed without locking, so context switching is very fast.
Communication	Inter-process communication requires the help of the operating system	Threads can directly read and write process data segments (such as global variables) to communicate.	Shared memory, message queue

Comparison of threads and processes

The comparison between threads and processes is as follows:

• The process is the unit of resource (including memory, open files, etc.) allocation , and the thread is  the unit of CPU scheduling ;

• The process has a complete resource platform, while the thread only enjoys essential resources, such as registers and stacks ;

• Threads also have three basic states : ready, blocked, and executed , and also have conversion relationships between states;

• Threads can reduce the time and space overhead of concurrent execution ;

For threads, threads can reduce overhead compared to processes, which is reflected in:

• The creation time of threads is faster than that of processes, because during the process of creation, processes also need resource management information, such as memory management information and file management information. However, during the creation process of threads, these resource management information will not be involved, but shared . they;

• The termination time of threads is faster than that of processes because threads release much fewer resources than processes ;

• Thread switching within the same process is faster than process switching because threads have the same address space (virtual memory sharing), which means that threads in the same process all have the same page table , so there is no need to switch page tables when switching. . For switching between processes, the page table must be switched when switching , and the page table switching process is relatively expensive;

• Since memory and file resources are shared between threads in the same process, data does not need to go through the kernel when transferring data between threads , which makes data interaction between threads more efficient;

How many threads can a process create, and what does it have to do with?

This depends on different systems:

• If it is a 32-bit system, the user-mode virtual space is only 3G. If the stack space allocated when creating a thread is 10M, then a process can only create about 300 threads at most.
• If it is a 64-bit system, the user-mode virtual space is as large as 128T. In theory, it will not be limited by the size of the virtual memory, but will be limited by the parameters or performance of the system.

The sizes of the following three kernel parameters will affect the upper limit of thread creation:

• */proc/sys/kernel/threads-max* , indicating the maximum number of threads supported by the system , the default value is  14553;

• */proc/sys/kernel/pid_max* represents the limit of the global PID number value of the system . Each process or thread has an ID. If the ID value exceeds this number, the process or thread will fail to be created. The default value is  32768;

• */proc/sys/vm/max_map_count* , which means limiting the number of VMA (virtual memory areas) that a process can have. I don’t know what it means specifically. Anyway, if its value is very small, it will also cause thread creation to fail. , the default value is  65530.

By the way, too many threads will cause a lot of time to be wasted on thread switching, which will have a negative impact on program running efficiency. Useless threads must be destroyed in time.

What is the difference between external interrupt and exception?

External interrupts are caused by events other than the execution of instructions by the CPU, such as I/O completion interrupts, which indicate that device input/output processing has been completed and the processor can send the next input/output request. In addition, there are clock interrupts, console interrupts, etc.

Exceptions are caused by internal events when the CPU executes instructions, such as illegal opcodes, address out-of-bounds, arithmetic overflow, etc.

The difference between synchronous and asynchronous

Synchronization: Doing only one thing at the same time.
Asynchronous: Multiple things can be done at the same time.

Advantages and disadvantages of synchronous and asynchronous

1. Synchronous execution efficiency will be relatively low and time-consuming , but it will help us control the process and avoid many uncontrollable unexpected situations;
2. Asynchronous execution is highly efficient and saves time , but will occupy more resources and also It is not conducive for us to control the process.

Asynchronous usage scenarios:

1. * Does not involve shared resources, or is read-only for shared resources, that is, non-mutually exclusive operations.
2. * There is no strict relationship in timing. 3. No
atomic operations are required, or atomicity can be controlled in other ways.
4. * Commonly used in IO. Operation and other time-consuming operations, because it affects the customer experience and performance
5. *Does not affect the main thread logic

Synchronous usage scenarios:

When not using asynchronous

Benefits of synchronization:

1. The synchronization process is usually simpler to process the results and can be processed nearby.
2. The processing of results by the synchronization process is always within the same context as the previous one.
3. The synchronization process can easily catch and handle exceptions.
4. Synchronization process is the most natural way to control the sequential execution of processes.

Advantages of asynchronous:

1. The asynchronous process can immediately return preliminary results to the caller .
2. The asynchronous process can delay the final result data to the caller, and during this period more additional work can be done, such as result recording, etc.
3. During the execution of the asynchronous process, the occupied threads and other resources can be released to avoid blocking , and the thread processing can be reacquired until the result is generated.
4. The asynchronous process can wait for the results of multiple calls to come out, and then return a unified result set to improve response efficiency.
 

How much do you know about the process thread model?

Multithreading

There are multiple threads in the same process. All threads share the memory space of the same process. Each thread also has its own independent stack space. Global variables defined in the process will be shared by all threads, and multiple threads are used by the CPU. The order of scheduling is uncontrollable, so access to critical resources requires special attention to safety.

First of all, the original sequentially executed program (not considering multi-process for the moment) can be split into several independent logical flows. These logical flows can independently complete some tasks (preferably these tasks are unrelated).

• Is there a sequential access sequence between threads (thread dependency)?

• Multiple threads share access to the same variable (synchronization mutual exclusion problem)

As the smallest unit of processor scheduling, thread scheduling only needs to save the thread stack, register data and PC, which is much less expensive than process switching.

There are many thread-related interfaces, and you mainly need to understand the meaning of each parameter and the meaning of the return value.

1. Thread creation and termination

   • background knowledge:

     Multiple functions in a file are usually executed in the order they appear in the main function. However, under a time-sharing system, we can make each function execute concurrently as a logical flow. The simplest way is to use multi-threading. Strategy. In the main function, the multi-thread interface is called to create threads. Each thread corresponds to a specific function (operation). This way, the functions in the main function can be executed in the order in which they appear, avoiding busy waiting. The interface for basic thread operations is as follows.

   • Related interfaces:

     • 创建线程：int pthread_create(pthread_t *tidp,const pthread_attr_t *attr, void *(start_rtn)(void),void *arg);

       To create a new thread, pthread and start_routine are indispensable. They are used to identify the thread and execution body entrance respectively. Others can be filled with NULL.

       • pthread: used to return the tid of the thread. The *pthread value is tid. The type pthread_t == unsigned long int.

       • attr: Pointer to the thread attribute structure, used to change the attributes of the created thread. Fill in NULL and use the default value.

       • start_routine: The first address of the thread execution function, pass in the function pointer.

       • arg: Pass function parameters through address passing. Here is an unsigned type pointer. You can pass the address of any type of variable. In the passed-in function, the type must be forced to the required type first.

     • Get the thread ID: pthread_t pthread_self();

       When called, the thread ID is printed.

     • Wait for the thread to end: int pthread_join(pthread_t tid, void** retval);

       Called by the main thread, waiting for the child thread to exit and recycle its resources, similar to wait/waitpid in the process to recycle the zombie process, the thread calling pthread_join will be blocked.

       • tid: Get the tid value through the pointer when creating a thread.

       • retval: Pointer to the return value.

     • End the thread: pthread_exit(void *retval);

       Child thread execution is used to end the current thread and pass the return value through retval, which can be obtained through pthread_join.

       • retval: Same as above.

     • Detach thread: int pthread_detach(pthread_t tid);

       It can be called by both the main thread and sub-threads. pthread_detach(tid) in the main thread and pthread_detach(pthread_self()) in the sub-thread. After the call, it is separated from the main thread. When the sub-thread ends, it will immediately recycle the resources.

       • tid: Same as above.

2. Thread attribute value modification

   • background knowledge:

     The thread attribute object type is pthread_attr_t, and the structure is defined as follows:

     typedef struct{
         int detachstate;    // 线程分离的状态
         int schedpolicy;    // 线程调度策略
         struct sched_param schedparam;    // 线程的调度参数
         int inheritsched;    // 线程的继承性
         int scope;    // 线程的作用域
         // 以下为线程栈的设置
         size_t guardsize;    // 线程栈末尾警戒缓冲大小
         int stackaddr_set;    // 线程的栈设置
         void *    stackaddr;    // 线程栈的位置
         size_t stacksize;    // 线程栈大小
     }pthread_attr_t;
     

• Related interfaces:

  Most of the parameters in the above structure include: pthread_attr_get() and pthread_attr_set() system call functions to set and obtain. Not listed here.

Application>Program=Process>Thread

The relationship between threads and processes: An application (APP) can have multiple processes, which are independent of each other. A process can start multiple threads, which is equivalent to division of labor to complete process tasks. For example, an auxiliary aiming program is a process that completes all tasks in the auxiliary aiming process; the camera task (in the auxiliary aiming code) is a thread, which is only responsible for reading the video stream from the camera (or local video) for use by other threads.

The difference between threads and processes: Each process has an independent virtual address space, which means that the data between processes are isolated from each other and cannot be accessed directly. Each thread of the same process shares the same virtual address space, that is, one thread can directly access variables in another thread (if the address is known).

Shared data: Shared data refers to data that may be accessed by multiple threads at the same time. It can be said that data sharing is the meaning of multi-threading, but it will also bring problems such as data competition. Data competition means that while a thread is reading a shared variable, one or more other threads modify the shared variable. This causes the intermediate state of the variable to be read, which may cause undefined errors.

Mutex locks: Mutex locks can maintain the order in which each thread accesses shared data. A mutex lock can only be acquired by one thread at a time. If other threads try to acquire the lock, they will be blocked until the lock is released. Get the lock. Note: The mutex lock does not actually lock the data, but only the "lock". In other words, if one thread uses the lock and another thread does not use the lock, or uses another lock, then the mutex lock will be invalid at this time and will not have the effect of preventing data competition. Function: Solve data competition, but using mutex locks can also cause deadlocks and other problems.

Summary: The process is the running program, the thread is the task in the program, the process is the basic unit of resource allocation, and the thread is CPU scheduling, or the smallest unit of program execution; single thread: there is only one execution path in the program; multi-threading : The program has multiple paths. Using multi-threading can improve the execution efficiency of the program.

Replenish:

1. Classification of concurrency: (1) Task concurrency: This is also commonly understood concurrency. A task is divided into multiple subtasks. (2) Data concurrency: Group data and perform the same operation on different groups of data.

2. What is thread safety?

If the running results of a multi-threaded program are predictable and the same as those of a single-threaded program, then it is "thread-safe".

3. How many methods are there to implement multi-thread synchronization and mutual exclusion? What are they?

1. Critical section : Access public resources or a piece of code through serialization of multiple threads. It is fast and suitable for controlling data access. Only one thread is allowed to access shared resources at any time.

2. Mutex: Designed to coordinate separate accesses to a shared resource. A mutex is very similar to a critical section. Only the thread that owns the mutex object has the permission to access the resource. The thread currently occupying the resource should hand over the mutex object it owns after the task is processed, so that other threads can obtain it. Access resources. Mutexes are more complex than critical sections. Because the use of mutual exclusion can not only achieve safe sharing of resources among different threads of the same application, but also achieve safe sharing of resources between threads of different applications.

3. Semaphore: Designed for controlling a resource with a limited number of users. When creating a semaphore, indicate both the maximum allowed resource count and the currently available resource count. Generally, the current available resource count is set to the maximum resource count. Each time an additional thread accesses the shared resource, the current available resource count will be reduced by 1. As long as the current available resource count is greater than 0, a semaphore signal can be sent. However, when the current available count decreases to 0, it means that the number of threads currently occupying resources has reached the maximum allowed number, and other threads cannot be allowed to enter. At this time, the semaphore signal will not be sent. After the thread finishes processing the shared resources, it increases the number of currently available resources by 1. The current available resource count can never be greater than the maximum resource count at any time.

4. Event: used to notify the thread that some events have occurred, thereby starting the subsequent task.

Note: Mutexes, semaphores, and events can all be used across processes to perform synchronous data operations.

Monitors and semaphores?

4. What are the similarities and differences between multi-thread synchronization and mutual exclusion, and under what circumstances should they be used respectively? for example

​ The so-called synchronization means that there is precedence. A more formal explanation is that " thread synchronization refers to a restriction relationship between threads. The execution of one thread depends on the message of another thread. When it does not get the message of another thread, You should wait until the message arrives." The so-called mutual exclusion is more formally explained as " thread mutual exclusion refers to the exclusivity of shared process system resources when each single thread accesses them. When there are several When several threads want to use a certain shared resource, only one thread is allowed to use it at any time. Other threads that want to use the resource must wait until the occupier releases the resource. Thread mutual exclusion can be regarded as a special Thread synchronization." means that they cannot be accessed at the same time, which is also a sequence issue, so mutual exclusion is a special synchronization operation.

For example, there is a global variable global. In order to ensure thread safety, we stipulate that only when the main thread modifies global, the next sub-thread can access global. This requires synchronization of the main thread and sub-threads, which can be achieved by key segments. When a child thread accesses global and another thread cannot access global, mutual exclusion is needed.

e. Which of the following multi-threaded operations on int type variable x need to be synchronized:

A. x=y; B. x++; C. ++x; D. x=1;

ABC, obviously, the writing of y and the reading of y by x must be synchronized. Both x++ and ++x need to know the previous value of x, so they must also be synchronized.

f. In multi-threading, the stack is private and the heap is public. The stack generally stores local variables, and programmers generally apply for and release data in the heap themselves.

g. In Windows programming, mutexes and critical sections are similar. Please analyze the main differences between the two.

1) A mutex is a kernel object, so it consumes more resources than a critical section, but it can be named and therefore can be accessed by other processes.

2) In terms of purpose, the critical section is to access public resources or a piece of code through serialization of multi-threads. It is fast and suitable for controlling data access. Mutexes are designed to coordinate separate accesses to a shared resource.

5. Three basic states and five basic operations of threads

Basic states: executing, ready, blocked.

Basic operations: derive, block, activate, schedule, end.

multi-Progress

Each process is the basic unit of resource allocation.

The process structure consists of the following parts: code segment, stack segment, and data segment. Code snippets are static binary code that can be shared by multiple programs.

In fact, after the parent process creates the child process, almost all parts of the parent and child processes are almost the same except the pid.

The parent and child processes share all data, but it does not mean that they operate on the same piece of data. When the child process reads and writes data, it will copy the common data again through the copy-on-write mechanism , and then use the copied data to Perform operations.

If the child process wants to run its own code segment, it can also reload the new code segment by calling the execv() function, and then it will be independent from the parent process.

When we execute a program in the shell, we first fork() a child process through the shell process and then reload the new code segment through execv().

1. Process creation and termination

   • background knowledge:

     There are two ways to create a process, one is created by the operating system and the other is created by the parent process. The process from computer startup to terminal execution program is: process No. 0 -> kernel process No. 1 -> user process No. 1 (init process) -> getty process -> shell process -> command line execution process. So when we execute an executable file through ./program on the command line, all the processes created are child processes of the shell process. This is why as soon as the shell is closed, the processes executed in the shell are automatically closed. From the shell process to creating other sub-processes, you need to pass the following interface.

   • Related interfaces:

     • Create process: pid_t fork(void);

       Return value: -1 on error; pid > 0 in parent process; pid == 0 in child process

     • End the process: void exit(int status);

       • status is the exit status, which is stored in the global variable S?, usually 0 means normal exit.

     • 获得PID：pid_t getpid(void);

       Returns the caller's pid.

     • Get the parent process PID: pid_t getppid(void);

       Returns the parent process pid.

   • Other additions:

     • Normal exit methods: exit(), _exit(), return (in main).

       The difference between exit() and _exit(): exit() is an encapsulation of __exit(). It will terminate the process and do related finishing work. The main difference is that the _exit() function closes all descriptors and cleans up the function. The stream will be refreshed, but exit() will refresh the data stream before calling the _exit() function.

       The difference between return and exit(): exit() is a function, but it has parameters. After execution, control is handed over to the system. If return is in the calling function, control is given to the calling process after execution. If return is in the main function, control is given to the system.

     • Abnormal exit methods: abort(), termination signal.

2. Linux process control

• Process address space (address space)

  Virtual memory provides each process with the illusion of exclusive use of the system's address space.

  Although the contents of each process' address space are different, they all have a similar structure. The bottom of the address space of the X86 Linux process is reserved for user programs, including text, data, heap, stack, etc. The text area and data area map the corresponding segments of the executable file on the disk to the virtual memory address space through memory mapping. middle.

  There are some "sensitive" addresses to note. For 32-bit processes, the code segment starts at 0x08048000. Starting from 0xC0000000 to 0xFFFFFFFF is the kernel address space. Normally, the code runs in user mode (using the user address space of 0x00000000 ~ 0xC00000000). When a system call, process switch and other operations occur, the CPU control register sets the mode bit and enters the internal mode. , in this state (superuser mode) the process can access all memory locations and execute all instructions.

  In other words, the address space of 32-bit processes is all 4G, but in user mode, only the lower 3G address space can be accessed. If you want to access the 3G ~ 4G address space, you can only enter the kernel mode.

• Process control block (processor)

  Process scheduling actually means that the kernel selects the corresponding process control block. The selected process control block contains basic information about a process.

• context switch

  The kernel manages all process control blocks, and the process control block records all status information of the process. Every process scheduling is a context switch. The so-called context is essentially the current running state, which mainly includes general registers, floating point registers, status registers, program counters, user stacks and kernel data structures (page table, process table, file table) wait.

  At the moment of process execution, the kernel can decide to preempt the current process and start a new process. This process is completed by the kernel scheduler. When the scheduler selects a process, it is said that the process is scheduled. This process changes the current state through context switching.

  A complete context switch usually means that the process originally ran in user mode, and then switched to kernel mode to execute kernel instructions due to system calls or time slices. After completing the context switch, it returned to user mode. At this time, it has switched to process B.

What is the difference between multi-processing and multi-threading? In other words, when should you use multi-threading and when should you use multi-processing?

• Frequent modification: multi-threading is preferred if frequent creation and destruction is required
• Calculation amount: Multi-threading is preferred for those that require a lot of calculations. Because  it consumes a lot of CPU resources and switches frequently , multi-threading is better.
• Correlation: If the correlation between tasks is relatively strong, use multi-threading , and if the correlation is relatively weak, use multi-process. Because data sharing and synchronization between threads is relatively simple.
• Multi-distribution: It may be extended to use multiple processes for multi-machine distribution , and multi-threads for multi-core distribution .

But in practice, what is more common is the combination of process and thread, which is not either/or.

Process communication method (under Linux and windows), thread communication method (under Linux and windows)

Process communication method

Name and method
Pipe: allows communication between one process and another process with which it has a common ancestor
Named pipe (FIFO): Similar to a pipe, but it can be used for communication between any two processes. Named pipes have corresponding file names in the file system. Named pipes are created through the command mkfifo or the system call mkfifo
Message Queue (MQ): A message queue is a connection table of messages, including POSIX message pairs and System V message queues. A process with sufficient permissions can add messages to the queue, and a process granted read permissions can read messages from the queue. The message queue overcomes the shortcomings of the small amount of information carried by the signal, the pipeline can only be an unformatted byte stream, and the limited buffer size;
Semaphore: Semaphore is mainly used as a means of synchronization between processes and between different threads of the same process;
Shared memory: It allows multiple processes to access the same memory space and is the fastest form of IPC available. **This is designed for other communication mechanisms to operate less efficiently. It is often used in conjunction with other communication mechanisms, such as semaphores, to achieve synchronization and mutual exclusion between processes.
Signal: Signal is a relatively complex communication method used to notify the receiving process that something has happened. In addition to being used for inter-process communication, the process can also send signals to the process itself.
Memory mapping (mapped memory): Memory mapping allows communication between any multiple processes. Each process using this mechanism achieves it by mapping a shared file into its own process address space.
Socket: It is a more general inter-process communication mechanism that can be used for inter-process communication between different machines.

Thread communication method

Name and meaning
Linux：
Signal: similar to signal processing between processes
Lock mechanism: mutex lock, read-write lock and spin lock
Condition variable: Unlock using notification method, used in conjunction with mutex lock
Semaphores: including unnamed thread semaphores and named thread semaphores
Windows：
Global variables: When multiple threads need to access a global variable, we usually add a volatile statement in front of the global variable to prevent the compiler from optimizing this variable.
Message message mechanism: There are two commonly used Message communication interfaces: PostMessage and PostThreadMessage. PostMessage is a thread that sends messages to the main window. PostThreadMessage is the communication interface between any two threads.
CEvent object: CEvent is an object in MFC. You can change the triggering state of CEvent to achieve communication and synchronization between threads. This is mainly a method to achieve direct synchronization of threads.

What are the methods of inter-process communication? Tell me everything you know

 Since the user space of each process is independent and cannot access each other, it is necessary to use the kernel space to achieve inter-process communication. The reason is very simple. Each process shares a kernel space.

The Linux kernel provides many methods of inter-process communication, the simplest of which is pipes. Pipes are divided into "anonymous pipes" and "named pipes".

As the name suggests, anonymous pipes have no name identifier. Anonymous pipes are special files that only exist in memory and do not exist in the file system. The " " vertical bar in the shell command |is an anonymous pipe. The communication data is an unformatted stream and has a limited size. , the communication method is one- way, and data can only flow in one direction. If you want two-way communication, you need to create two pipes. Furthermore, anonymous pipes can only be used for inter-process communication with a parent-child relationship . The life cycle of anonymous pipes It is established when the process is created and disappears when the process terminates.

Named pipes break through the limitation that anonymous pipes can only communicate between related processes. Because the premise of using named pipes is to create a device file of type p in the file system, then unrelated processes can use this device file. communication. In addition, whether it is an anonymous pipe or a named pipe, the data written by the process is cached in the kernel . When another process reads the data, it will naturally obtain it from the kernel. At the same time, the communication data follows the first-in-first-out principle and does not support lseek. Class file location operations.

The message queue overcomes the problem that the data in pipeline communication is an unformatted byte stream . The message queue is actually a "message linked list" stored in the kernel . The message body of the message queue is a user-defined data type. When sending data, It will be divided into independent message bodies. Of course, when receiving data, it must also be consistent with the data type of the message body sent by the sender, so as to ensure that the data read is correct. The speed of message queue communication is not the most timely. After all, every writing and reading of data requires a copy process between user mode and kernel mode.

Shared memory can solve the overhead caused by the data copy process between user mode and kernel mode in message queue communication . It directly allocates a shared space, and each process can directly access it . It is as fast and convenient as accessing the process's own space. It needs to fall into the kernel state or system call, which greatly improves the speed of communication and enjoys the reputation of being the fastest inter-process communication method. However, convenient and efficient shared memory communication brings new problems. Multiple processes competing for the same shared resource will cause data confusion.

Then, a semaphore is needed to protect the shared resources to ensure that only one process can access the shared resources at any time. This method is mutually exclusive access. The semaphore can not only achieve mutual exclusion of access, but also achieve synchronization between processes . The semaphore is actually a counter, which represents the number of resources. Its value can be controlled through two atomic operations, namely  P operation and V operation .

The one with a very similar name to a semaphore is called a signal . Although their names are similar, their functions are completely different. Signals are an asynchronous communication mechanism . Signals can interact directly between application processes and the kernel . The kernel can also use signals to notify user space processes of what system events have occurred. The main sources of signal events are hardware sources (such as keyboard Cltr+C). And software sources (such as kill command), once a signal occurs, the process has three ways to respond to the signal: 1. Perform the default operation, 2. Capture the signal, 3. Ignore the signal . There are two signals that cannot be caught and ignored by the application process, namely  SIGKILL and  SIGSTOP. This is for the convenience of us to end or stop a process at any time.

The communication mechanisms mentioned earlier all work on the same host. If you want to communicate with processes on different hosts, Socket communication is required . Socket is actually not only used for communication between different host processes, but also for communication between local host processes. It can be divided into three common communication methods according to the type of created Socket, one is based on the TCP protocol, and the other is One is a communication method based on UDP protocol, and the other is a local inter-process communication method.

The above is the main mechanism of inter-process communication.

What is the difference between signal and semaphore?

• Signals: A way of handling asynchronous events . Signals are a relatively complex communication method used to notify the receiving process that a certain event has occurred . In addition to being used for processes, signals can also be sent to the process itself.

• Semaphore: A mechanism for inter-process communication to handle synchronization and mutual exclusion . It is a facility used in a multi-threaded environment. It is responsible for coordinating various threads to ensure that they can use public resources correctly and reasonably.

How is shared memory implemented?

The mechanism of shared memory is to take out a piece of virtual address space and map it to the same physical memory . In this way, what is written by this process can be seen by another process immediately, without copying and passing it around, which greatly improves the speed of inter-process communication.

Can you introduce some typical locks?

read-write lock

• Multiple readers can read at the same time
• Writers must be mutually exclusive (only one writer is allowed to write, and readers and writers cannot write at the same time)
• Writers take priority over readers (once there is a writer, subsequent readers must wait, and writers are given priority when waking up)

mutex lock

Only one thread can own the mutex lock at a time, other threads can only wait

A mutex lock actively gives up the CPU and goes to sleep when the lock grab fails and then wakes up when the lock status changes. The operating system is responsible for thread scheduling. In order to wake up the blocked thread or process when the lock status changes, you need to The lock is managed by the operating system, so the mutex lock involves context switching during the locking operation.

condition variable

An obvious disadvantage of a mutex is that it has only two states: locked and non-locked. Condition variables make up for the shortcomings of mutex locks by allowing threads to block and wait for another thread to send a signal. They are often used together with mutex locks to avoid race conditions. When a condition is not met, the thread often unlocks the corresponding mutex lock and blocks the thread and then waits for the condition to change. Once another thread changes the condition variable, it will notify the corresponding condition variable to wake up one or more threads that are blocked by this condition variable. In general, mutex locks are mutual exclusion mechanisms between threads, and condition variables are synchronization mechanisms.

spin lock

If the incoming thread cannot obtain the lock, the incoming thread will not give up the CPU time slice immediately, but will keep trying to acquire the lock in a loop until it is obtained. If other threads occupy the lock for a long time, then spin is wasting the CPU on useless work. However, spin locks are generally used in scenarios where the locking time is very short, and the efficiency is relatively high at this time.

deadlock

It may cause two threads to wait for each other to release the lock . Without external force, these threads will keep waiting for each other and will be unable to continue running. In this case, a deadlock occurs .

Deadlock will occur only if the following four conditions are met at the same time :

• Mutually exclusive condition; multiple threads cannot use the same resource at the same time .

• Hold and wait conditions; Thread A will not release the resource 1 it already holds while waiting for resource 2 .

• Inalienable condition; it cannot be acquired by other threads before it is used up.

• Loop waiting condition; the order in which two threads obtain resources forms a circular chain

To avoid the deadlock problem, you only need to break one of the loop conditions. The most common and feasible method is to use the orderly allocation method of resources to break the loop waiting condition . Thread A and thread B always apply for the resources they want in the same order (resource 1 first, then resource 2).

Approach

There are mainly four methods:

• Ostrich strategy: solutions that take no measures will achieve higher performance
• Deadlock detection: Execute the pstack command multiple times to view the function calling process of the thread , compare the results multiple times, and confirm which threads have not changed and are waiting for the lock, then it is most likely caused by a deadlock problem.
• Deadlock recovery: recovery by preemption; recovery by rollback; recovery by killing the process
• Deadlock prevention:

1. Breaking mutual exclusion conditions: For example, spooling printer technology allows several processes to output at the same time. The only process that actually requests a physical printer is the printer daemon.
2. Destruction request and hold conditions: One way to achieve this is to specify that all processes request all resources they need before starting execution.
3. Destruction of non-deprivation conditions: Allowed to seize resources
4. Destroy the cycle request waiting: Give resources a unified number, and the process can only request resources in the order of numbers.

• Deadlock avoidance

banker's algorithm

We can think of the operating system as a banker. The resources managed by the operating system are equivalent to the funds managed by the banker. The process's request to allocate resources to the operating system is equivalent to the user's loan to the banker.

To ensure the safety of funds, bankers stipulate:

(1) A customer can be accepted when his maximum demand for funds does not exceed the banker's existing funds;

(2) Customers can borrow in installments, but the total number of loans cannot exceed the maximum demand;

(3) When the banker's existing funds cannot meet the loan amount that the customer still needs, the loan payment to the customer can be postponed, but the customer can always get the loan within a limited time;

(4) When the customer obtains all the funds required, all funds must be returned within a limited time.

The operating system allocates resources to the process according to the rules set by the banker. When a process applies for resources for the first time, it tests the process's maximum demand for resources. If the existing resources of the system can meet its maximum demand, it will be allocated according to the current application amount. resources, otherwise allocation is deferred. When the process continues to apply for resources during execution, it is first tested whether the number of resources requested by the process exceeds the total amount of resources remaining. If it exceeds, the allocation of resources will be refused. If it can be met, the resources will be allocated according to the current application amount, otherwise the allocation will be postponed.

How to recycle threads? What are the methods?

• Wait for the thread to end:  int pthread_join(pthread_t tid, void** retval);

  Called by the main thread, waiting for the child thread to exit and recycle its resources, similar to wait/waitpid in the process to recycle the zombie process, the thread calling pthread_join will be blocked.

  • tid: Get the tid value through the pointer when creating a thread.

  • retval: Pointer to the return value.

• End the thread:  void pthread_exit(void *retval);

  Child thread execution is used to end the current thread and pass the return value through retval, which can be obtained through pthread_join.

  • retval: Same as above.

• Detach thread:  int pthread_detach(pthread_t tid);

  It can be called by both the main thread and sub-threads. pthread_detach(tid) in the main thread and pthread_detach(pthread_self()) in the sub-thread. After the call, it is separated from the main thread. When the sub-thread ends, it will immediately recycle the resources.

  • tid: Same as above.

When the terminal exits, what will happen to the process running in the terminal?

When the terminal exits, it will send SIGHUP to the corresponding bash process. After receiving this signal, the bash process will first send it to the process under the session. If the program does not do special processing for the SIGHUP signal, then the process will exit when the terminal is closed.

How to make a process run in the background

(1) Just add & after the command. In fact, this puts the command into a job queue.

(2) ctrl + z suspends the process, use jobs to view the serial number, and use bg % serial number to run the process in the background

(3) nohup + &, standard output and standard error will be redirected to the nohup.out file by default, and all hangup (SIGHUP) signals will be ignored.

(4) Run + setsid in front of the command to make its parent process program the init process and not be affected by the HUP signal

(5) Put the command + & in () brackets, or the process will not be affected by the HUP signal

Daemon, zombie and orphan processes

daemon

Refers to a process that runs in the background and has no control terminal connected to it. It is independent of the control terminal and performs certain tasks periodically. Most servers in Linux are implemented as daemon processes, such as web server processes http, etc.

Key points for creating a daemon process:

(1) Let the program execute in the background. The method is to call fork() to generate a child process, and then cause the parent process to exit.

(2) Call setsid() to create a new session. Control terminals, login sessions, and process groups are usually inherited from the parent process. The daemon process needs to get rid of them by calling setsid(). After the call is successful, the process becomes the new session group leader and process group leader, and is the same as the original one. Login sessions, process groups, and controlling terminals are detached.

(3) Prohibit the process from reopening the control terminal. After the above steps, the process has become a terminalless session leader, but it can reapply to open a terminal. To prevent this from happening, you can make the process no longer the session leader. Create a new child process through fork() again, causing the process that called fork to exit.

(4) Close file descriptors that are no longer needed. The child process inherits open file descriptors from the parent process. If it is not closed, system resources will be wasted, the file system where the process is located will not be able to be unmounted, and unpredictable errors will occur. First get the highest file descriptor value, then use a loop program to close all file descriptors from 0 to the highest file descriptor value.

(5) Change the current directory to the root directory.

(6) The file creation mask inherited by the child process from the parent process may deny certain permissions. To prevent this, use unmask(0) to clear the mask word to zero.

(7) Process the SIGCHLD signal. For server processes, when a request comes, a child process is often generated to handle the request. If the child process waits for the parent process to capture the status, the child process will become a zombie process (zombie), occupying system resources. If the parent process waits for the child process to end, it will increase the burden on the parent process and affect the concurrency performance of the server process. Under Linux, you can simply set the operation of the SIGCHLD signal to SIG_IGN. In this way, no zombie process will be generated when the child process ends.

Orphan process

If the parent process exits first and the child process has not exited, then the parent process of the child process will become the init process. (Note: Any process must have a parent process).

If a parent process exits while one or more of its child processes are still running, those child processes will become orphan processes. The orphan process will be adopted by the init process (process number is 1), and the init process will complete the status collection work for them.

zombie process

If the child process exits first and the parent process has not exited, then the child process must wait until the parent process captures the exit status of the child process before it actually ends, otherwise the child process will become a zombie process at this time.

The purpose of setting up a zombie process is to maintain information about the child process so that the parent process can obtain it at some point in the future. This information includes at least the process ID, the process's termination status, and the CPU time used by the process, so this information can be obtained when the parent process that terminates the child process calls wait or waitpid. If a process terminates and has child processes in the zombie state, the parent process IDs of all its zombie child processes will be reset to 1 (init process). The init process that inherits these child processes will clean them up (that is, the init process will wait for them, thus removing their zombie status).

How to avoid zombie processes?

• Notify the kernel through signal (SIGCHLD, SIG_IGN) that it does not care about the end of the child process and will be recycled by the kernel. If you do not want the parent process to hang, you can add a statement to the parent process: signal(SIGCHLD,SIG_IGN); indicating that the parent process ignores the SIGCHLD signal, which is sent to the parent process when the child process exits.

• The parent process calls functions such as wait/waitpid to wait for the child process to end. If no child process has exited wait, the parent process will be blocked. waitpid can make the parent process return immediately without blocking by passing WNOHANG.

• If the parent process is very busy, you can use signal to register a signal processing function, and call wait/waitpid in the signal processing function to wait for the child process to exit.

• By calling fork twice. The parent process first calls fork to create a child process and then waitpid waits for the child process to exit . The child process then forks a grandchild process and then exits. In this way, after the child process exits, it will be waited for recycling by the parent process. For the grandson process, its parent process has already exited, so the grandson process becomes an orphan process. The orphan process is taken over by the init process . After the grandson process ends, init will wait for recycling.

The first method ignores the SIGCHLD signal. This is a technique often used to improve the performance of concurrent servers because concurrent servers often fork many child processes. After the child processes terminate, the server process needs to wait to clean up resources. If the processing method of this signal is set to ignore, the kernel can transfer the zombie child process to the init process for processing, saving a large number of zombie processes from occupying system resources.

Parent process, child process, process group, job and session

parent process

A process that has created one or more child processes

child process

The new process created by fork is called a child process. The function is called once, but returns twice (on the child and on the parent) . The difference between the two returns is that the return value of the child process is 0, while the return value of the parent process is the process id of the new process (child process). The reason for returning the child process ID to the parent process is: because a process can have more than one child process, there is no function that allows a process to obtain the process IDs of all its child processes. For the child process, the reason why fork returns 0 to it is because it can call getpid() at any time to get its own pid; it can also call getppid() to get the id of the parent process. (Process ID 0 is always used by the swap process, so a child process cannot have a process ID of 0).

After fork, the operating system will copy a child process that is exactly the same as the parent process. Although it is a father-son relationship, from the perspective of the operating system, they are more like brothers . The two processes share the code space , but the data space is independent of each other. , the content in the data space of the child process is a complete copy of the parent process, and the instruction pointer is exactly the same. The child process has the position where the parent process is currently running (the program counter pc value of the two processes is the same, that is to say, the child process is from fork execution starts at the return point), but there is one difference. If the fork is successful, the return value of fork in the child process is 0, and the return value of fork in the parent process is the process number of the child process. If the fork is unsuccessful, the parent process will return an error. .

The child process inherits from the parent process:

1. Qualifications of the process (real/effective/saved user IDs (UIDs) and group IDs (GIDs))

2. environment

3.Stack

4.Memory

5. Process group number

Unique:

1. Process number;

2. Different parent process numbers (Translator's Note: The parent process number of the child process is different from the parent process number of the parent process. The parent process number can be obtained by the getppid function);

3. Resource utilizations are set to 0

process group

A process group is a collection of multiple processes, among which there must be a group leader whose process PID is equal to the PGID of the process group. As long as a process exists in a process group, the process group exists, regardless of whether its leader process terminates.

Operation

The shell is divided into front and backends to control not processes but jobs or process groups .

A foreground job can be composed of multiple processes , and a background job can also be composed of multiple processes. The shell can run a foreground job and any number of background jobs. This is called job control.

Why can I only run one foreground job?

Answer: When we start a new job in the foreground , the shell is brought to the background , so the shell cannot continue to accept our instructions and parse and run them. But if the foreground process exits, the shell will be brought to the foreground, and it can continue to accept our commands and parse and run them.

The difference between a job and a process group : If a process in the job creates a child process, the child process does not belong to the job. Once the job ends, the shell brings itself to the foreground ( the child process still exists , but the child process does not belong to the job ). If the original foreground process still exists (the child process has not been terminated), it will automatically become a background process group.

session

A session is a collection of one or more process groups . A session can have one controlling terminal. Opening a window in xshell or WinSCP creates a new session.

Several ways to terminate the process

1. The natural return of the main function . return 2. The calling exitfunction belongs to the function library of c. 3. The calling _exitfunction belongs to the system call. 4. The calling abortfunction terminates abnormally and sends the SIGABRT signal to the calling process. 5. Accept signals that can cause the process to terminate: ctrl+c (^C), SIGINT (SIGINT interrupts the process)

The difference between exit and _exit

When memory swapping occurs, which processes are given priority? Can you talk about it?

Blocked processes can be swapped out first; low-priority processes can be swapped out; in order to prevent low-priority processes from being swapped out soon after being transferred into memory, some systems will also consider the residence time of the process in the memory... (Note: PCB will be resident in memory and will not be swapped out of external memory)