Four characteristics of an operating system?
Concurrency: Execution of multiple programs within the same period of time (different from parallelism, parallelism refers to multiple events at the same time, multi-processor systems can make programs execute in parallel)
Sharing: resources in the system can be shared by multiple concurrently executing threads in memory
Virtual: Virtualize a physical entity into multiple
Asynchronous: The system process is executed in a stop-and-go manner (not all at once), and it is unpredictable when and how fast the process will move forward
process thread
A process refers to an application program running in memory, and each process has its own independent memory space.
A thread is an execution unit smaller than a process. It is an independent control flow in a process. A process can start multiple threads, and each thread executes different tasks in parallel.
The difference between process and thread is as follows :
- Scheduling: Process is the basic unit of resource management, and thread is the basic unit of program execution.
- Switching: Thread context switching is much faster than process context switching.
- Owning resources: A process is an independent unit that owns resources. Threads do not own system resources, but can access resources belonging to processes.
- System overhead: When creating or canceling a process, the system must allocate or reclaim system resources for it, such as memory space, I/O devices, etc. The overhead paid by the OS is significantly greater than the overhead when creating or canceling threads, and the overhead of process switching It is also much greater than the overhead of thread switching.
This article has been included in the Github warehouse, which includes computer foundation, Java foundation, multithreading, JVM, database, Redis, Spring, Mybatis, SpringMVC, SpringBoot, distributed, microservices, design patterns, architecture, school recruitment and social recruitment sharing, etc. Core knowledge points, welcome to star~
If you can't access Github, you can access the gitee address.
Concurrency and Parallelism
Concurrency means that multiple tasks will be processed within a period of time; but at a certain moment, only one task is executing. Single-core processors can do concurrent. For example, there are two processes Aand B, Aafter running a time slice, switch to B, and Bthen switch to after running a time slice A. Because the switching speed is fast enough, it is macroscopically shown that multiple programs can be run at the same time for a period of time. The most comprehensive Java interview site
Parallelism means that multiple tasks are executing at the same time. This requires a multi-core processor to complete. On a micro level, multiple instructions can be executed at the same time. Different programs are run on different processors. This is physically multiple processes at the same time.
Advantages of multithreading over single threading
1. Concurrency improves program execution efficiency
2. Improve CPU utilization, and you can switch threads to execute when accessing memory
3. Faster response speed, you can have a dedicated thread to monitor user requests and a dedicated thread to process requests. For example, the listening thread and the working thread are two threads, so the listening thread is responsible for monitoring, and the worker is responsible for the work. When a user request is detected, the request is immediately transferred to the working thread for processing, and the listening thread continues to monitor.
What is a coroutine?
A coroutine is a lightweight thread in user mode.
The coroutine is not managed by the operating system kernel, but completely controlled by the user program. The benefit of this is that the performance is greatly improved, and it does not consume resources like thread switching.
A coroutine can be understood as a function that can suspend execution. It has its own register context and stack. When the coroutine schedule is switched, the register context and stack are saved to other places. When switching back, the previously saved register context and stack are restored. There is basically no kernel switching overhead for direct operation of the stack, and global variables can be accessed without locking. , so context switching is very fast.
What is the difference between threads and coroutines?
1. Threads are preemptive, while coroutines are non-preemptive, so the user needs to release the usage right to switch to other coroutines. Therefore, only one coroutine has the right to run at the same time, which is equivalent to the ability of a single thread.
2. Threads are resources of coroutines. Coroutines use thread resources indirectly through the executor (Interceptor) that can be associated with any thread or thread pool.
process communication
There are several ways to communicate between processes:
1. Pipeline communication
Anonymous pipe ( pipe ): pipe is a half-duplex communication method, data can only flow in one direction, and can only be used between processes with kinship. Process affinity usually refers to the parent-child process relationship.
The well-known pipe is a half-duplex communication method, and data can only flow in one direction.
2. Message queue
3. Shared memory . Shared memory is the fastest method of IPC, and it is specifically designed to run inefficiently when other methods of interprocess communication run. It is often used in conjunction with other communication mechanisms, such as semaphores, to achieve synchronization and communication between processes.
4. Semaphore . A semaphore is a counter that can be used to control access to shared resources by multiple processes. It is often used as a locking mechanism to prevent other processes from accessing shared resources while a process is accessing the resource. Therefore, it is mainly used as a means of synchronization between processes and between different threads in the same process.
What is deadlock?
Deadlock refers to a phenomenon in which two or more threads wait for each other due to competition for resources during execution. If there is no external force, they will not be able to advance.
As shown in the figure below, thread A holds resource 2, and thread B holds resource 1. They both want to apply for the resource held by the other party at the same time, so the two threads will wait for each other and enter a deadlock state.
[External link picture transfer failed, the source site may have an anti-theft link mechanism, it is recommended to save the picture and upload it directly (img-E8WXcqiK-1681546229546)(http://img.topjavaer.cn/img/deadlock.png)]
The following example illustrates thread deadlock, and the code comes from the beauty of concurrent programming.
Finally, I would like to share with you a Github warehouse, which has more than 300 classic computer book PDFs compiled by Dabin, including C language, C++, Java, Python, front-end, database, operating system, computer network, data structure and algorithm, machine learning , programming life , etc., you can star it, next time you look for a book directly search on it, the warehouse is continuously updated~
public class DeadLockDemo {
private static Object resource1 = new Object();//资源 1
private static Object resource2 = new Object();//资源 2
public static void main(String[] args) {
new Thread(() -> {
synchronized (resource1) {
System.out.println(Thread.currentThread() + "get resource1");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(Thread.currentThread() + "waiting get resource2");
synchronized (resource2) {
System.out.println(Thread.currentThread() + "get resource2");
}
}
}, "线程 1").start();
new Thread(() -> {
synchronized (resource2) {
System.out.println(Thread.currentThread() + "get resource2");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(Thread.currentThread() + "waiting get resource1");
synchronized (resource1) {
System.out.println(Thread.currentThread() + "get resource1");
}
}
}, "线程 2").start();
}
}
The code output is as follows:
Thread[线程 1,5,main]get resource1
Thread[线程 2,5,main]get resource2
Thread[线程 1,5,main]waiting get resource2
Thread[线程 2,5,main]waiting get resource1
Thread A acquires the monitor lock on resource1 via synchronized(resource1) and passes Thread.sleep(1000). Let thread A sleep for 1s in order to allow thread B to be executed and then acquire the monitor lock of resource2. After the sleep of thread A and thread B is over, they both start to request the other party's resources, and then the two threads will fall into a state of waiting for each other, which will cause a deadlock.
How does the deadlock occur? How to avoid it?
Four necessary conditions for deadlock to occur :
-
Mutual exclusion: a resource can only be used by one process at a time
-
Request and Hold: When a process is blocked due to requesting resources, the obtained resources are not released
-
No deprivation: The resources obtained by the process cannot be forcibly deprived before they are used
-
Circular waiting: cyclically waiting for resources between processes
Ways to avoid deadlock :
- Mutual exclusion conditions cannot be destroyed, because locking is to ensure mutual exclusion
- Apply for all resources at one time, avoiding threads occupying resources and waiting for other resources
- When a thread that occupies some resources further applies for other resources, if the application cannot be obtained, it will actively release the resources it occupies
- Apply for resources in sequence
What are the process scheduling strategies?
-
First come, first served : Non-preemptive scheduling algorithm, scheduling according to the order of requests. It is good for long jobs, but it is not good for short jobs, because short jobs have to wait for the previous long jobs to be executed before they can be executed, and long jobs need to be executed for a long time, resulting in too long waiting time for short jobs. In addition, it is not good for
I/Ointensive processes, because such processesI/Ohave to be requeued after each operation. -
Shortest job first : A non-preemptive scheduling algorithm that schedules in the order with the shortest estimated running time. Long jobs may starve to death and are in a state of waiting for short jobs to finish. Because if short jobs keep coming, long jobs will never be scheduled.
-
Shortest Remaining Time First : A preemptive version of shortest job first, scheduled in order of remaining run time. When a new job arrives, its overall running time is compared with the remaining time of the current process. If the new process requires less time, suspend the current process and run the new process. Otherwise the new process waits.
-
Time slice rotation : Arrange all ready processes into a queue according
FCFSto the principle of ,CPUand allocate time to the queue leader process each time it is scheduled, and this process can execute a time slice. When the time slice is used up, the timer sends a clock interrupt, and the scheduler stops the execution of the process and sends it to the end of the ready queue, while continuing to allocateCPUtimeThe efficiency of the time slice rotation algorithm has a lot to do with the size of the time slice: because process switching must save the process information and load the information of the new process, if the time slice is too small, the process switching will be too frequent. would take too much time. And if the time slice is too long, real-time performance cannot be guaranteed.
-
Priority scheduling : Assign a priority to each process and schedule according to the priority. To prevent low-priority processes from never waiting to be scheduled, the priority of waiting processes can be increased over time.
What are the states of a process?
A process has a total 5of three states, namely creation, ready, running (execution), termination, and blocking.
- The running state is that the process is
CPUrunning on it. In a uniprocessor environment, at most one process is running at a time. - The ready state means that the process is already in a state of being ready to run, that is, the process has obtained
CPUall the required resources except the resource, andCPUcan run once obtained. - A blocked state is when a process is suspended while waiting for an event, such as waiting for a resource to become available or to
I/Ocomplete. Even ifCPUidle, the process cannot run.
Running state→blocking state : It is often caused by waiting for peripherals, waiting for resource allocation such as main memory, or waiting for manual intervention.
Blocked state→ready state : the waiting condition has been met, and it can run after being allocated to the processor.
Running state→ready state : It is not due to its own reasons, but because of external reasons, the process in the running state gives up the processor, and then it becomes the ready state. For example, the time slice is used up, or there is a higher priority process to preempt the processor, etc.
Ready state→Running state : The system selects a process in the ready queue to occupy the processor according to a certain strategy, and it becomes the running state at this time.
How to understand the memory fragmentation in the operating system?
Memory fragmentation is usually divided into internal fragmentation and external fragmentation:
- Internal fragmentation is due to the use of fixed-size memory partitions. Internal fragmentation occurs when a process cannot fully use the fixed memory area allocated to it. Usually internal debris is difficult to completely avoid
- External fragmentation is a memory area that cannot be utilized by a process because some unallocated continuous memory area is too small to satisfy the memory allocation request of any process.
What is the solution ?
The commonly used memory allocation method is segment page memory allocation. Divide memory into segments, and each segment into fixed-size pages. Through the page table mechanism, the pages in the segment do not have to be continuously in the same memory area.
Virtual Memory
Virtual memory is a memory system that has the function of calling in requests and can logically expand the memory capacity. Virtual memory has three characteristics: multiplicity, interchangeability, and virtuality. It can call programs in multiple times. Memory allows larger user programs to be executed in a smaller user space, so more processes can be executed concurrently, thereby improving system throughput. When a page fault occurs, either a segment or a page can be loaded, depending on how the memory is managed. Virtuality means the mapping of virtual memory and physical memory.
Under Linux, the process cannot directly read and write the physical address of the memory, but can only access the [virtual memory address]. The operating system will put the virtual memory address -> physical address.
Virtual memory solves the problem of loading larger applications with limited memory space, moving data back and forth between memory and disk as needed.
Through the form of the segment page table, a continuous memory space in the virtual memory is mapped to the main memory, and the program segments in the main memory space may not be continuous.
What is pagination?
Divide the memory space into blocks of equal size and fixed , as the basic unit of main memory. Because the program data is stored in different pages, and the pages are discretely distributed in the memory, a page table is needed to record the mapping relationship to realize the mapping from the page number to the physical block number.
Accessing the memory data in the paging system requires two memory accesses (one is to access the page table from the memory, find the specified physical block number from it, and add the offset in the page to obtain the actual physical address; the second is to obtain the actual physical address based on the first time) The physical address access memory to fetch data).
What is segmentation?
Paging is to improve memory utilization, and segmentation is to meet some logic requirements of programmers when writing code (such as data sharing, data protection, dynamic linking, etc.).
In segmented memory management, the address is two-dimensional, one-dimensional is the segment number, and two-dimensional is the address within the segment; the length of each segment is different, and the addressing of each segment starts from 0 . In segment management, each segment is allocated with continuous memory, but segments are allocated discretely, so there is also a mapping relationship between logical addresses and physical addresses, corresponding to the segment table mechanism.
What is the difference between paging and segmentation?
- Paging is transparent to the programmer, but segmentation requires the programmer to explicitly divide each segment.
- The address space of paging is a one-dimensional address space, and segmentation is two-dimensional.
- The page size is immutable, and the segment size can be changed dynamically.
- Paging is mainly used to implement virtual memory to obtain a larger address space; segmentation is mainly to enable programs and data to be divided into logically independent address spaces and to facilitate sharing and protection.
Page Replacement Algorithm
Why page replacement:
Because the application program is loaded into the memory multiple times, a page fault will definitely occur after running for a certain period of time. During the address mapping process, if the page to be accessed is found not in the memory, a page fault interrupt will be generated. At this time, the operating system must select a page in the memory to move it out of the memory to make room for the page to be transferred in. The rule for choosing which page to eliminate is the page replacement algorithm
Several page replacement algorithms:
Optimal replacement algorithm (ideal) : Replace pages that will not be accessed in the current page for the longest time in the future
First in, first out : Eliminate the page that was loaded the earliest
LRU has not been used for the longest time : each page has a t to record the last time the page was accessed until now, and the page with the largest t value is replaced every time it is replaced (implemented with registers or stacks)
Clock algorithm clock (also known as the recently unused algorithm NRU): the page is set to access, linking the pages into a circular list, each page has an access bit 0/1, 1 means another chance to be rescued, the next cycle When the pointer points to it, this replacement can be exempted, but the access position will be set to 0, which means that it should be replaced if it encounters a loop pointer next time. The access bit is set to 1 when the page is accessed. When the page is replaced, if the access bit of the current pointer is 0, replace it, otherwise set this value to 0, and loop until the page with the access bit of 0 is encountered.
Improved clock algorithm : Add a modification bit to the clock algorithm, firstly replace pages with both the access bit and the modification bit at 0, and then replace pages with the access bit at 0 and the modification bit at 1.
Least used algorithm LFU : Set the register to record the number of page visits, and replace the one with the least number of current visits each time.
user mode and kernel mode
Kernel mode: The cpu can access all data in the memory, including peripheral devices, such as hard disks, network cards, and the cpu can also switch itself from one program to another.
User mode: Only limited access to memory, and no access to peripheral devices, the ability to occupy the CPU is deprived, and CPU resources can be obtained by other programs.
The biggest difference is that permissions are different. Programs running in user mode cannot directly access operating system kernel data structures and programs.
Why are there these two states?
The kernel is fast but has limited resources, and there are not many processes that can be controlled, so it needs the assistance of a slower user mode. However, in order to prevent the user mode from being maliciously used, the permissions of the user mode programs are limited.
It is necessary to restrict access between different programs to prevent them from obtaining memory data of other programs, or obtaining data of peripheral devices and sending them to the network. The CPU is divided into two permission levels - user mode and kernel mode.
when to convert
1. System call :
Initiated by the user process. The user-mode process requests to use the service program provided by the operating system to complete the work through the system call. For example, fork() is to execute a system call to create a new process.
The user program uses the system call, the system call will be converted to the kernel mode and call the operating system
2. An exception occurs :
It will switch from the currently running process to the kernel-related program that handles this exception
3. Interruption of peripheral equipment:
All programs run in the user mode, but when reading data from the hard disk or inputting from the keyboard, only the operating system can do these things, and the program needs to request the operating system to perform these operations in the name of the program. At this time, the user mode program switches to the kernel mode.
What is a buffer overflow? What's the harm?
Buffer overflow means that when the computer fills the buffer with data beyond the capacity of the buffer itself, the overflowed data is overwritten on the legal data.
There are two hazards:
- Program crashes, resulting in a denial of service
- Jump and execute a piece of malicious code
The main cause of buffer overflow is that the user input is not carefully checked in the program.
I/O multiplexing
IO multiplexing means that once the kernel finds that one or more IO conditions specified by a process are ready to be read, it notifies the process. IO multiplexing is applicable to the following occasions :
- I/O multiplexing must be used when clients are dealing with multiple descriptors (typically interactive input and network sockets).
- While it is possible, but rare, for a client to handle multiple sockets simultaneously.
- If a TCP server handles both listening sockets and connected sockets, I/O multiplexing is generally used.
- If a server needs to process both TCP and UDP, I/O multiplexing is generally used.
- If a server handles multiple services or multiple protocols, I/O multiplexing is generally used.
- Compared with multi-process and multi-thread technology, the biggest advantage of I/O multiplexing technology is that the system overhead is small, the system does not need to create processes/threads, and does not need to maintain these processes/threads, thus greatly reducing the system overhead.