Operating system principle OS review summary

Written in front: this articleContinually updated, Review and summarize the study of operating system principles. Related learning reference content has been listed below,The current level is limited, if you have any suggestions, please correct me!
References

1. Operating system principle [Tianjin University, Li Gang, 2013] (Station B, Youku)
2. https://github.com/CyC2018/CS-Notes
3. Public Account: Java Builders-Article: Processes and Threads for Busy People

1. Process Management

1.1 process

The CPU is a multi-channel system, pseudo-parallel (single core single CPU)

Process: an instance of a program that is being executed (with "life")

What is the difference between program and process ?

1. Program is just the static text.
2. Process is an dynamicentity being executed and has lifecycle.

A process consists of three parts:

• Process Control Block (PCB) or Process Table Entry
• Program (Text code)
• Data

Except for the ancestor process in Unix, everything else is fork ("clone") itself. Shared body (code, data), head is different (Block)

Process States:

• The scheduler (scheduler) initiates 2, 3 (conversion between R and R)
  Running and Ready together are called Runnable (runnable state) The
  scheduler (scheduler) can also choose one from n Runnable to run
• Running takes the initiative 1
• 4, Blocked wakes up and re-queues
• Scheduler has a high priority

process termination:
Normal exit （自杀）
Error exit
Fatal error
Killed by another process （他杀）

1.2 threads

The process is the smallest unit of resource allocation, and the threads in the process share the resources they own.
Thread is the smallest unit of scheduling.
Scheduling internal threads running processes
Implementing Threads in User Space (user mode, kernel mode)
thread pool (to be updated)

1.3 Scheduling

• CPU intensive process
• IO-intensive process

Batch processing system: high CPU utilization rate.
Interactive system: low CPU utilization rate. Care about response time.
Real-time system: respond immediately when the conditions are met (industrial boiler)

Scheduling of batch system:

• First Come First Served
  a first-come, first-served
• Shortest Job First
  short operating priority
• Shortest Remaining-time Next
• High Response-ratio First
  最高响应比优先
  turnaround time/executing time
  = (waiting time + executing time)/executing time
  = 1 + waiting time / executing time

Scheduling of interactive systems:

• Time slice rotation
• Priority scheduling
• Multi-level feedback queue (combination of time slice round-robin scheduling algorithm and priority scheduling algorithm)

1.4 Race Conditions and Semaphore, PV and Semaphore

Critical resources: can be shared and used, but cannot be used at the same time (eg. printer).
Critical regions: code that uses critical resources

Race conditions greater than 2 processes:

• Mutex (Mutex): grab critical resources, and have nothing to do with each other
• Synchronization: there is an inherent logic sequence (such as strict rotation)

Conditions required to avoid race condition:

• No two processes nay be simultaneously insider their critical regions.
• No assumptions may be made about speeds or the number of CPUs.
• No process running outside its critical region may block other processes.
• No process should have to wait forever to enter its critical region.

The resolution of race conditions is also divided into two categories:

• Busy waiting
• Sleep and wakeup (in general, save resources) Sleep and Wakeup

Busy waiting program: TSL (Test and Set Lock). Keywords of the law:"Spin Lock" "Busy waiting" "Atomicity, that is, uninterruptible"

// TSL代码
enter_region
	// MOVE REGISTER, #1
	// XCHG REGISTER, LOCK |原子性操作
	
	TSL REGISTER, LOCK  |复制lock到寄存器，并将lock置为1 (这句话实现上面两句话) 原子性操作
	
	CMP REGISTER, #0    | lock等于0吗?
	
	JNE enter_region    |如果不等于0，已上锁，再次循环
	
	RET                 |返回调用程序，进入临界区

leave_region

	MOVE LOCK, #0       |置lock为0
	
	RET                 |返回调用程序

Semaphore can be operated by PV operation only.
signalIt can be directly assigned only when initial value is assigned, others can only be operated with P and V

（note, The semaphore has a waiting queue part, not just an integer part)

The physical meaning of the integer part in Semaphore:

• Positive: the number of resources available. (the number of available resources represented by the semaphore)
• 0 Zero: no resource available, no waiting process. (There is no idle resource, and no waiting process)
• Negative: the number of process waiting to use the resources. (The number of processes waiting to use the resources)

The source words of P and V cannot be interrupted (P drops, V rises). Source language: very fast, very fast, uninterrupted, small amount.

// 方案1：PV in Sleep abd Wakeup
P(Semaphore s)
{
    
    
	s = s - 1;
	if (s < 0)
	{
    
    
		add to the semaphore's queue and sleep;
	}
}

V(Semaphore s)
{
    
    
	s = s + 1;
	if (s <= 0)
	{
    
    
		wake up the waiting process in the semaphore's queue;
	}
}

// 方案2：PV in Busy Waiting
P(Semaphore s)
{
    
    
	while (!s>0)
	{
    
    
		yield the CPU;
	}
	s--;
}

V(Semaphore s)
{
    
    
	s++;
}

The above two PVs can be used as examples of printers to consider.
Option 1, I occupied, go to bed, and then someone wakes up.
Option 2, I’m busy waiting, keep testing whether there are resources.
TubeMonitors
can guarantee that they are not executed at the same time. Use a compiler to compile the monitor. For example, if you add synchronized to one piece of code in Java and also add synchronized to another piece of code, the two pieces of code will not run at the same time.

1.5 Classic synchronization problem of process synchronization

Synchronization problem: first check, then modify, it may be interrupted in the middle, the previous check is invalid, and errors will occur (TSL and PV both avoid such problems)

1.5.1 The Producer-Consumer Problem

#define N 100
typedef int semaphore;
semaphore mutex = 1; // 生产者消费者互斥访问临界区
semaphore empty = N; // 当前空格子的个数（生产者的）
semaphore full = 0;  // 满格子的个数（消费者的）
					 // empty、full可以叫做通行证
void producer(void)
{
    
    
	int item;
	
	while(TRUE)
	{
    
    
		item = produce_item();
		p(&empty);
		p(&mutex);
		insert_item(item);
		v(&mutex);
		v(&full);
	}
}

void consumer(void)
{
    
    
	int item;
	
	while(TRUE)
	{
    
    		
		p(&full);      //这两个p颠倒 + 生产者两个p也颠倒，会发生死锁
		p(&mutex);
		item = remove_item();
		v(&mutex);
		v(&empty);
		consume_item(item);
	}
}
              

1.5.2 Dining Philosophers Problem

// Mutex保护操作
int state[N]
semaphore mutex=1; // 在这种用PV的解决问题的类型中，int这种全局变量，需加Mutex一下

1.5.3 The Readers and Writers Problem

Reader-first reader-writer issues:
1. Writers and readers mutually exclusive access to file resources.
2. Multiple readers can access file resources at the same time.
3. Only one writer is allowed to access file resources.

typedef int semaphore;
void reader(void)
{
    
    
	while(TRUE)
	{
    
    
		down(&mutex);
		rc = rc + 1;
		if (rc == 1) {
    
     down(&db); }
		up(&mutex);
		read_data_base();
		down(&mutex);
		rc = rc - 1;
		if (rc == 0) {
    
     up(&db); }
		up(&mutex);
	}
	void writer(void)
	{
    
    
		while(TRUE)
		{
    
    
			down(&db);
			write_data_base();
			up(&db);
		}
	}
}

1.6 IPC Inter-Process Communications

Process synchronization: control multiple processes to execute in a certain order;
process communication: transfer information between processes.
Process communication is a means, and process synchronization is a purpose. It can also be said that in order to achieve the purpose of process synchronization, it is necessary to allow processes to communicate and transmit some information required for process synchronization.

• pipeline
• FIFO (named pipe)
• message queue
• signal
• Shared storage
• Socket

1.7 deadlocks

Four conditions for deadlock of preemptible and non-preemptible resources (eg. printer) :

1. Mutual exclusion condition
2. Hold and wait condition (partially owned, expect more)
3. No preemption condition
4. Circular wait condition

Resolve deadlock (ignore, detect, avoid, prevent)

• Just ignore the problem
  Ostrich algorithm, Assuming no deadlock
• Detection and recovery.
  Detection: Matrix, resource directed graph. Recovery: Kill young processes
• Dynamic avoidance by careful resource allocation
  banker algorithm (security sequence, security status)
• Prevention by structurally negating one of the four required conditions
  . The process sends this to some queue (spool), waiting for this critical resource is equivalent to sending to the queue and thinking that the task is complete. (Eg. Printer, send mail). This method and the ostrich are commonly used and widely used, and other resources cannot be determined or expected for the process.

1.8 Supplement

makefile
system call: fork()…and others
multithreading

2. Memory management

2.1 Overview

Primary Memory (aka RAM)
Secondary Memory (ie. hard disk)
CPU only deals with memory, not hard disk

• Process Memory Layout
  

Code segment (Text Segment): generally read-only and immutable.
Data Segment: (Static) The size is fixed.
Heap Storage: Variable size. The feature is random access.
Stack Segment: Variable size. The characteristic is first in, last out. The high address is fixed, and it expands or shrinks toward the low address.
Environment Variables area (Environment Variables): Variables set by the outside, generally the operating system.

2.2 Partition

Partition managementIt is to put the whole and continuous into the memory space, which brings about the problem of not being able to use the discontinuous small space (fragment).

• Fixed-Partition example Fixed partition (with internal fragmentation)
• Variable-Partition example Variable partition (with external fragmentation)

Partition management positioning requires base register Base register

Memory Manager Strategies MM strategy

• Overlaying technology (small memory, unable to fit all)
• Swapping exchange technology

Paging management can be distributed.

2.3 Paging

Virtual Memory
The purpose of virtual memory is to expand the physical memory into a larger logical memory, so that the program can get more available memory.
In order to better manage memory, the operating system abstracts memory into an address space. Each program has its own address space, this address space is divided into multiple blocks, each block is called a page. These pages are mapped to physical memory, but do not need to be mapped to contiguous physical memory, nor do they need all pages to be in physical memory. When the program references a page that is not in the physical memory, the hardware performs the necessary mapping, loads the missing part into the physical memory and re-executes the failed instruction.

Page table

• Page page (for example, a page 4K, you can set it yourself)
• Page frame
  

Management page table:

1. Accelerate
   Translation Lookaside Buffers, TLB. It is placed in the high-speed cache, also called fast table (so the outside is called slow table).
   A TLB to speed up paging.
   If you put all the page tables in memory, it takes a lot of time to access the page tables, so put some items in the Cache to improve the table lookup speed.
2. Smaller
   hierarchical page table Multilevel Page Tables
   the large page table into small portions, because usually a lot of empty page table entry
   inverted page table Inverted Page Tables
   Hash

Two points of attention:
Interruption: execute the next statement at the interrupted point after the scene is restored.
Page fault interruption: Return to execute the interrupted statement after the scene is restored.

Problems with paging:

1. The remaining pages are not full
2. Mechanical paging by size,A section of program may include main and sub-functions. It is not good to divide these. For example, a page has main and some sub-functions.
3. Same as 2, when encountering loop code, there is only one piece of memory left, which is constantly accessed (extremely special cases), and a lot of time is changing pages instead of executing statements, which is called "bumps"

2.4 Segmentation

Segmentation management solves the problem that paging management is not logically built, but it introduces the problem of partition management (fragmentation).
Segmentation, and then paging within the segment, that is, segmentation with Paging

2.5 page replacement algorithm page replacement

The real physical memory is exhausted and some things need to be replaced. If a problem arises, who should be replaced?
When the program is running, if the page to be accessed is not in the memory, a page fault interrupt occurs and the page is loaded into the memory. If the memory has no free space at this time, the system must call a page from the memory to the disk swap area to make room. Page replacement algorithm and cacheEliminatedThe strategy is similar. You can think of memory as a disk cache. In the cache system, the size of the cache is limited. When a new cache arrives, a part of the existing cache needs to be eliminated, so that there is space to store the new cache data. The main goal of the page replacement algorithm is to minimize the page replacement frequency (or the lowest page fault rate).

Flag bits in the page table:

• P bit (protection, in memory or not)
• R bit (referenced, has it been used)-----cleared to 0 periodically
• M bit (modified, is it dirty)

Page replacement algorithm:

The content of the page replacement algorithm is best to read online courses to understand, easy to forget

1. FIFO, First In First Out
   The page selected to be swapped out is the page entered first. The algorithm will swap out the pages that are frequently accessed, thereby increasing the page fault rate.
2. Second Chance Algorithm
   FIFO algorithm may replace frequently used pages. To avoid this problem, a simple modification is made to the algorithm: when the page is accessed (read or written), the R bit of the page is set to 1. When replacement is needed, check the R bit of the oldest page. If the R bit is 0, then the page is old and unused and can be replaced immediately; if it is 1, clear the R bit to 0, put the page at the end of the linked list, and modify its load time so that It is just like it was loaded, and then it continues to search from the head of the linked list.
3. The Clock Page Replacement Algorithm
   second chance algorithm needs to move pages in the linked list, which reduces efficiency. The clock algorithm uses a circular linked list to connect the pages, and then uses a pointer to point to the oldest page.
4. (Least Recently used, LRU) Page Replacement Algorithm is the
   least recently used (the most powerful ones in the past are not necessarily eliminated, they have priority, the algorithm is the best, but it is almost impossible to achieve)
   This algorithm takes into account time and efficiency, in the recent time, use The least.
   1. N*N matrix, set the row to 1, the column to 0
   2. Simulating LRU in software (aging algorithm)
   (00000000) 8 bits, set to 1 after use, move back in the next clock cycle, and finally smaller than the position Important, because the calculated number is large) Compared with the authentic LRU disadvantage: a cycle is used several times without counting, and it will be topped after 8 cycles.
5. Working set page replacement Working set algorithm
   If R is 1, it has been used for a long time. R is 0, it is useless for a period of time recently, than age, throw away. (Record time point)
6. The WSClock Page replacement algorithm

Not Frequency Used
page stealer, if the page space is less than Min, start stealing (changing) until it reaches Max. Do not let it be too full, and ensure that the amount of free memory is between min and max.

2.6 Supplement

Comparison of paging and segmentation:

• Transparency to the programmer: Paging is transparent, but segmentation requires the programmer to display the division of each segment.
• Whether the size can be changed: The size of the page is immutable, and the size of the segment can be changed dynamically.
• Reasons for emergence: paging is mainly used to implement virtual memory to obtain a larger address space; segmentation is mainly to enable programs and data to be divided into logically independent address spaces and to facilitate sharing and protection.

3. storage management

Memory—memory
External storage—storage The
data in the memory is lost when power is off, and some of it needs to be placed in the external storage .

3.1 files and directories

The files include:

• Normal file
• Catalog file
• Special files (device files, hardware devices, logical devices. Manage external devices as files. others: sockets)

The file structure includes (file names and attributes in unix are stored separately):

• File name (a file can have multiple names (shared files))
• Properties (the judgement file has several usesinodeWithout name)
• content

file permission file permission (after the Linux summary review article will summarize the relevant content here)

The directory records the correspondence between the file name and the inode

The Unix File System: multi-level index

3.2 file systems

File system:

3.3 disks

Components of storage (from small to large):

• Files
• Directories
• File systems
• Physical storage & logical storage
• Logical Volume Manager (LVM)

Surface, column (track, also called track), sector-------->Locate to data

Disk scheduling algorithm: The
factors affecting the time to read and write a disk block are:

• Rotation time (the spindle rotates the disk surface to make the head move to the appropriate sector)
• Seek time (braking arm moves to make the head move to the appropriate track)
• Actual data transfer time

Among them, the seek time is the longest, so the main goal of disk scheduling is to make the average seek time of the disk the shortest.

Algorithms include:
elevator algorithm
...
...
...

LVM

3.4 Supplement

Redundant Array of Independent Disk.
RAID: faster, more reliable, mirrored
RAID0, RAID1, RAID5...

RAM disks in-memory disks (for speed, such as Cache)

4. Device Management Input/Output Devices

Interrupts ------->Modern computers must have
DMA, Direct Memory Access, used to control some high-speed transmission
Buffering cache

5. Character Set

ASCII (American Standard Code for Information Interchange), 7 bits, 7 bits (8 bits per byte)
GBK: National standard code
UTF8: to ensure the same size of ASCII, but also to avoid problems such as truncation of a half of the word, extended Other language codes. The widest range.

Written at the end:
Later, I will continue to update the learning and summary of operating system-related knowledge.
Linux-related will also summarize an article
The current level is limited, if you have any suggestions, please correct me!