System architecture design professional skills·operating system

现在的一切都是为将来的梦想编织翅膀，让梦想在现实中展翅高飞。
Now everything is for the future of dream weaving wings, let the dream fly in reality.

Click to enter the series of articles directory

1. Operating system overview

Types of operating systems: batch, time-sharing, real-time, network operating systems, and distributed operating systems.

The operating system has five major functions: processor management, storage management, device management, file management and job management. Regardless of any type of operating system, there is such an allocation.

Most modern operating systems have two working states: core state and user state. Our general applications work in user mode, while the kernel module and the most basic operating system core work in core mode.

The structure of the operating system can be divided into unordered structure, hierarchical structure, object-oriented structure, symmetric multiprocessing structure and microkernel structure.

2. Process management

2.1 Process concept

A process is a process in which a program runs on a data collection. It is an independent unit for resource allocation and scheduling in the system. It consists of three parts: program block, process control block (PCB) and data block.

PCB, PCB is the only sign of the existence of a process, an integral part of the process, and a record-type data structure. The content includes process identifier, status, location information, control information, queue pointer (linking processes in the same status), priority, on-site protection zone, etc.

Processes and Procedures

The difference between process and program: A process is an execution process of a program. Without a program, there is no process.

Program is a static concept, while process is a dynamic concept. It is generated by creation and dies due to cancellation after completing the task; process is an independent unit of the system for resource allocation and scheduling, but program is not.

Processes and Threads

There are two basic attributes of a process: an independent unit that can own resources, and a basic unit that can independently schedule and allocate resources.
Generally, a process contains multiple threads.

2.2 Process status

Three-state diagram : three states: ready, running, and blocked. When ready, it waits for a time slice to run. Blocking means that the process has not been fed resources. When resources are available, it reaches the ready state.

Five state diagram : running state, active ready state, quiescent ready state, active blocking state and quiescent blocking state. In addition to the running state, the blocking state is divided into static blocking and active blocking, and the ready state is also divided into static ready and active ready, so there are five states in total.

The process management in a certain computer system adopts the three-state model. The following organization method adopts the index method. There are 2 running processes, 3 ready processes, and 4 blocked processes.

2.3 Process synchronization and mutual exclusion

How to distinguish synchronization and mutual exclusion?

Mutual exclusion , thousands of troops crossing a single-plank bridge, competition for similar resources

Synchronization , speed differences, stopping and waiting under certain circumstances, cooperative relationship between processes

Mutual exclusion mainly talks about the competitive relationship between similar resources

During the execution of a process, it may have to stop and wait for other processes. This is synchronization (there will be similar multiple-choice questions)

Multiple processes share a printer, typically mutually exclusive

After consumers consume goods, producers produce goods before they can be put into the market, which is a synchronous relationship.

2.4 PV operation

P is Passeren in Dutch and V is Verhoogo in Dutch.

PV operation is a common method to achieve process synchronization and mutual exclusion.

The P operation means applying for a resource, which is responsible for obtaining the resource and blocking. The V operation means releasing a resource.

P operations and V operations are low-level communication primitives and are inseparable during execution.

The PV operation consists of the P operation primitive and the V operation primitive (the primitive is an uninterruptible process).

The meaning of PV operation: We use semaphores and PV operations to achieve process synchronization and mutual exclusion.

Semaphore: It is a special variable (global), usually represented by a capital S.

In most cases (without prior explanation), the initial value of the semaphore S will be considered to be 1, and 1 represents the initial resource allocation call.

To operate the semaphore, the specific definition is as follows:

P operation (request to allocate a resource):

Decrement the value of the semaphore S by 1, that is, S = S - 1.
If S >= 0, the process continues to execute, otherwise the process enters the waiting state.

V operation (release a resource):

Add 1 to the value of the semaphore S, that is, S = S + 1.
If S > 0, the process continues to execute. Otherwise, it means that there are some processes in the waiting queue waiting for the resource and need to wake up and wait.

2.5 Precursor graph

The forward graph is a directed acyclic graph, consisting of nodes and directed edges. The nodes represent the operations of each program segment, and the directed edges between nodes represent the predecessor relationship between the operations of the two program segments. This kind of diagram can be used to describe the execution sequence relationship between multiple programs or processes.

Precursor graph marking rules: From small to large, all outflows from nodes are V operations, and all inflows are P operations. The arrow represents a semaphore S.

2.5 Deadlock

Process management is the core of the operating system, but if it is not designed properly, deadlock problems will occur. If a process is waiting for something that cannot happen, the process is deadlocked. And if one or more processes deadlock, it will cause a system deadlock.

How to prevent deadlock: sequential resource allocation (ordered resource allocation method), banker's algorithm.

Deadlock resource calculation

Deadlock calculation formula:

There are m shared resources and n processes. The maximum number of resources required by each process is w. Then only when m > n * (w - 1), there will be no deadlock.

​ That is, m is at least = n * (w - 1) + 1

Typical example: The system has three processes A, B, and C. Each of these three processes requires 5 system resources. What is the minimum number of resources that a system has if it is impossible to have a deadlock?

According to the deadlock calculation formula, 3 * (5 - 1) = 12, the system requires at least 12 + 1 = 13 resources.

2.6 Banker’s Algorithm

The principles of banker's algorithm for allocating resources:

(1) When the maximum demand for resources by a process does not exceed the number of resources in the system, the process can be admitted
(2) The process can request resources in installments, but the total number of requests cannot exceed the maximum demand
(3) When the system currently When the resources cannot satisfy the number of resources that the process still needs, the request for the process can be postponed, but the process can always get the resources within a limited time.

3. Storage management

3.1 Logical address and physical address

Logical address relative address, the address generated by the CPU, the logical address is used internally and for programming, and the address is not unique
Physical address absolute address, the address loaded into the memory address register, the real address of the memory unit address
relocation, the address in the program The process of converting a virtual address (logical address) into a real address (physical address) of memory

3.2 Type

The main purpose of storage management is to solve the problem of multiple users using main memory. Its storage management solutions mainly include partition storage management, paging storage management, segmented storage management, segmented page storage management and virtual storage management.

Partition storage management:

Paging storage management : basic paging
segmented storage management : basic segmented
segment paging storage management : basic segment paging
virtual storage management : request paging, request segmentation, request segment paging, corresponding to basic paging, segmentation, segment paging

3.3 Partition storage (continuous space)

Partitioned storage management divides the user area of ​​the main memory into several areas, each area is assigned to a user job, and restricts them to run only in their own area.

3.4 Page storage/paging storage (non-contiguous space)

Paged storage == paging storage

Divide the address space of a process (program) into several equal-sized areas, called pages.

Correspondingly, the main memory (memory) space is divided into several physical blocks of the same size as the page, called blocks or page frames.

When allocating main memory to a process, several pages in the process are loaded into multiple non-contiguous blocks.

Advantages: high utilization, small fragmentation, simple allocation and management.

Disadvantages: Increased system overhead; jitter may occur.

Logical address = page number + page address, physical address = page frame number + page address

Page storage (details): Divide the program and memory into blocks of the same size, and transfer the program into the memory in units of pages. Calculation
steps:
1. Split the logical address into: page number + intra-page address
2. Page number = Logical address/page size In-page address = Logical address% page size
3. Use page table to map page number and physical block number
4. Physical address = Physical block number * Page size + In-page address

The basic unit of page size/block size is B bytes. Pay attention to the unit conversion.
The difference between binary, decimal and hexadecimal. Pay attention to the simple algorithm.

Mainly examine page storage, convert logical address to physical address, pay attention to the simple algorithm of each base

3.5 Segmented storage/segmented storage (non-contiguous space)

Segmented storage == segmented storage

In a segmented storage management system, each segment is allocated a continuous partition, while each segment in the process can be discretely allocated to different partitions of the main memory.

A segment mapping table is established for each process in the system, referred to as "segment table". Each segment occupies an entry in the table, which records the starting address of the segment in main memory (also called the "base address") and the length of the segment. When the process is executing, it looks up the segment table to find the main memory area corresponding to each segment.

Segmented storage: The logical space is divided according to the natural segments in the user's job, and then transferred into the memory. The lengths of the segments can be different.

Advantages: Multi-programs share memory, and modifications to each program do not affect each other.

Disadvantages: low memory utilization and large waste of memory fragmentation.

The main inspection of segment storage is to determine whether the logical address is legal (segment number + offset). The offset cannot exceed the segment length.

"Address out of bounds" occurs when converting "logical address" to "physical address"

3.6 Segment page storage management (non-contiguous space)

The basic principle of the segment paging system is to first divide the entire main memory into equal-sized storage blocks (page frames), divide the user program into several segments according to the logical relationship of the program, and assign a segment name to each segment, and then Divide each segment into several pages and allocate them discretely in page frame units. In a segment-page system, the address structure consists of three parts: segment number, page number within the segment, and address within the page.

Segmented page storage: a combination of segmented and paged storage. First segment, then page. A program has several segments, and each segment can have several pages. The size of each page is the same, but the size of each segment is different.

Advantages: little space waste, easy storage sharing, easy storage protection, and dynamic connection.

Disadvantages: Due to the increase in management software, the complexity and overhead also increase, and the required hardware and occupied content also increase, which greatly reduces the execution speed.

3.7 Fast table

The fast table is a small-capacity associative memory (Associative Memory), which is composed of a cache. It is fast and can guarantee parallel searches based on the content from the hardware. It is generally used to store the page numbers of the few active pages that are currently accessed most frequently. .

Fast table: stores the page table in the Cache; slow table: stores the page table in the memory.

3.8 Virtual storage management

In the storage management scenario described earlier, each job must be allocated enough space to hold all the information. When the main memory space cannot meet the job requirements, the job cannot be loaded into the main memory for execution. If a job is only partially loaded into the main memory, it can start running. The remaining part is temporarily left on the disk and loaded into the main memory when needed. This can effectively utilize the main memory space. From the user's perspective, the main memory capacity of the system will be much larger than the actual main memory capacity. People call such memory virtual memory.

3.9 Page replacement algorithm

• Optimal replacement algorithm OPT
• Random elimination algorithm RAND
• First in first out algorithm FIFO, may "jitter"
• The algorithm LRU, which has not been used for the longest time, will not "jitter". It uses the theoretical basis of "locality principle" to eliminate the elements that have been accessed for the longest time.
• Least frequently used page replacement LFU, eliminating elements with the least recent access frequency
• Clock page replacement algorithm, round robin algorithm, not recently used page replacement algorithm NUR

4. Document Management

4.1 Index file structure

Files must be logically continuous and can be physically dispersed.

The logical structure of the file is convenient for users to use; the physical structure of the file is how it is stored in the physical device.

This part often examines the calculation and conversion of logical numbers and physical numbers, the specific type of index used, calculation of file length, etc.

Direct index range : number of index blocks * index block size

Level one indirect index range : (disk data block (physical disk block) size/address item size) * index block size

Secondary indirect index range : (disk data block (physical disk block) size / address item size) 2 * index block size

Third-level indirect index range : (disk data block (physical disk block) size / address item size) cubed * index block size

4.2 Bitmap

Bitmap method . This method is to create a bitmap on the external memory to record the usage of the file memory. Each bit corresponds to only one physical block on the file storage, and the values ​​0 and 1 represent free and occupied respectively.

The bitmap uses a binary bit to represent the usage of a disk block in the disk. When the value is "0", it means that the corresponding disk block is free; when it is "1", it means that it has been allocated. Some systems use "0" as the allocated mark of the disk block and "1" as the free mark. (They are essentially the same. They both use one bit to mark two states: free and allocated.) All disk blocks on the disk have a binary bit corresponding to them. In this way, all disk blocks have The corresponding bits form a set, called a bitmap. Bitmap is a very commonly used structure and is widely used in indexing, data compression, etc.

4.3 Tree directory structure

5. Equipment management

5.1 Data transmission control method

Device input/output (I/O) management methods:

The workload of the CPU becomes less and less from top to bottom, and the efficiency becomes higher and higher from top to bottom.

Partition storage management

• Program control mode [ Serial work of CPU and I/O ]: It is divided into two modes: unconditional transmission and program query mode. The method is simple and has low hardware overhead, but the I/O capability is not high, which seriously affects the CPU utilization. In program query mode, the CPU must constantly test the status port of the I/O device .
• Program interrupt mode [ Parallel work of CPU and I/O ]:
  Compared with the program control mode, the interrupt mode improves the response speed of transmission requests because the CPU does not need to wait. The CPU obtains the device status by calling the interrupt control device, and the device notifies the CPU of status changes, obtains data, etc. through interrupts. After the interrupt ends, the CPU will continue executing the previous program__.
• DMA (Direct Memory Access) [ Parallel work of CPU and I/O ]:
  Set up to achieve high-speed, batch data exchange between main memory and peripherals. DMA mode is more efficient than program control mode and interrupt mode. The DMA controller directly exchanges batch data with the memory. The CPU only controls the start and end. The transmission process does not require CPU intervention.
  DMA hardware is completed directly without any intervention from the CPU. It is suitable for fast I/O devices and responds to DMA after a bus of the CPU ends .
• Channel mode [ Parallel work of CPU and I/O ]:
  There is a dedicated channel processor in the host computer, with less CPU intervention. The corresponding channel program and channel status word are stored in the memory, and the CPU issues a startup command to the channel processor. Regardless, the channel processor executes the channel program and controls the device to initiate an interrupt after transmitting all data and hand it over to the CPU for processing, and the device is attached to the channel.
• Input-output processor (IOP) [ parallel work of CPU and I/O ]:
  an independent system with its own memory, instruction bear, and interrupt. It is dedicated to the control of input and output devices of large-scale and efficient systems, using shared memory, etc. to communicate with the host Exchange information. There may be multiple channels connected to the device in the input and output processor.

5.2 I/O management software

• Hardware, complete specific I/O operations.
• Interrupt handler: wakes up the device driver after I/O is completed
• Device driver, setting registers, checking device status
• Device-independent I/O layer, device name resolution, blocking process, buffer allocation
• User-level I/O layer (user process), issues I/O calls.

6. SPOOLing technology

SPOOLing is the abbreviation of Simultaneous Peripheral Operation On-Line (ie, external device online parallel operation). It is a technology about how slow character devices exchange information with the computer host, often called "spooling technology".

SPOOLing technology (a technology commonly used in virtual input and output devices) is first put into the disk buffer and then into the device area. It opens up a response area on the disk, so the buffer is external memory.

SPOOLing technology transforms a dedicated printer into a printer that can be shared by multiple users.

SPOOLing technical features:

(1) Input well and output well, two storage areas opened up on the disk, used to store data input by I/O devices and data output by user programs to the device (2) Input buffer area and output buffer area,
in Two buffers opened in memory. The input buffer user temporarily stores the data sent from the input device and then sends it to the input well; the output buffer temporarily stores the data sent from the output well and then transmits it to the output device
(3) The input process and the output process are responsible for transmitting input and output. Data
(4) improves the I/O speed, and alleviates the contradiction between the speed mismatch between the host and the peripheral device by operating the input well and the output well (5) The device is not directly
associated with the user process
(6) Implements virtual devices, many Two processes use a logical device at the same time, and each process thinks it has exclusive use of the device.

7. Microkernel

• Microkernel is to slim down the operating system and retain only the most basic functions. The ordinary operating system kernel includes five parts: process, storage, device, file, and task management. In the microkernel operating system , it has been tailored, and the file system, communication, and peripheral management are all placed in the operating system. The foreigner went to deal with it.
• Most modern operating systems have two working states, namely core state and user state . Generally, applications work in user mode, while kernel modules and the most basic operating system core work in core mode. Because in the microkernel, there are fewer core state functions, and many operations that originally belong to the operating system are implemented in the user state.
• The security and reliability of microkernel systems have been improved . Microkernel operating systems are widely used in distributed applications, and it has become the biggest advantage of microkernel operating systems.
• Of course, in a microkernel operating system, a task needs to be completed by constantly switching directly between the user mode and the kernel mode. Frequent switching is naturally not very efficient.

Place the traditional operating system code at a higher level and remove as many things as possible from the operating system, leaving only the smallest core, which is called a microkernel (C/S structure).

	substance	advantage	shortcoming
single core	All functions such as graphics, device drivers and file systems are implemented in the kernel and run in the kernel state and the same address space.	Reduce the system overhead of inter-process communication and status switching to achieve higher operating efficiency	The kernel is huge, takes up a lot of resources and is difficult to trim. System stability and security are not good
microkernel	Only basic functions are implemented, and the graphics system, file system, device driver and communication functions are placed outside the kernel.	The core is refined and easy to cut and transplant. The system service program runs in the user address space. The system has high reliability, stability and security and can be used in distributed systems.	User state and kernel state need to be switched frequently, resulting in system efficiency that is not as good as a single kernel

Click to enter the series of articles directory