computer system
big job
Topic Program Life-Hello's P2P
Professional Artificial Intelligence (2+x)
student number 202111****
class 21wl***
student Wen**
Instructor Wu Rui
School of Computer Science and Technology
May 2023
Summary
As every programmer knows, "Hello, world!" is the bedrock of the programming world. This article tells the story of a lifetime of the word "Hello". First, we wrote a source code file called hello.c. Then, it is preprocessed to generate a file called hello.i. Next, we compile it into an assembly language file named hello.s. Then, we convert the assembly file into a relocatable object file hello.o. Next, we use the linker ld to combine hello.o and the system target file to create an executable target file, namely hello.
When we run the program, we enter the command "./hello shell". The program first checks whether the input command is a built-in operation, if not, it starts calling the fork function to create a new process. Then, use the execve function to load hello into the memory, and the CPU controls the operation of the logic flow of the program, including interrupt, context switching and exception handling. Finally, the process ends and is recycled by the parent process, which marks the end of hello's "life".
Through this process, we understand the life cycle of hello in the computer, from source code to executable file, to the creation and end of the process. This process is what programmers go through all the time in the programming world, and it is the basis on which they build software.
Keywords: preprocessing; compilation; assembly; link; process; storage; program life
Table of contents
Chapter 1 Overview - 4 -
1.1 Introduction to HELLO - 4 -
1.2 Environment and tools - 4 -
1.3 Intermediate results - 4 -
1.4 Summary of this chapter - 4 -Chapter
2 Preprocessing - 5 -
2.1 The concept and function of preprocessing - 5 -
2.2 Preprocessing commands under UBUNTU - 5 -
2.3 Analysis of preprocessing results of HELLO - 5 -
2.4 Summary of this chapter - 5 -
Chapter 3 Compilation - 6 -
3.1 The concept and function of compiling - 6 -
3.2 Compiling commands under UBUNTU - 6 -
3.3 Compilation result analysis of HELLO - 6 -
3.4 Summary of this chapter - 6 -
Chapter 4 compilation - 7 -
4.1 The concept and function of compilation - 7 -
4.2 Compilation commands under UBUNTU - 7 -
4.3 Relocatable target ELF Format - 7 -
4.4 Analysis of the result of HELLO.O - 7 -
4.5 Summary of this chapter - 7 -
Chapter 5 Link - 8 -
5.1 The concept and function of link - 8 -
5.2 Link command under UBUNTU - 8 -
5.3 Executable target Format of file HELLO - 8 -
5.4 Virtual address space of HELLO - 8 -
5.5 Analysis of link relocation process - 8 -
5.6 HELLO Execution Process - 8 -
5.7 HELLO Dynamic Link Analysis - 8 -
5.8 Chapter Summary - 9 -
Chapter 6 HELLO Process Management - 10 -
6.1 Concept and Function of Process - 10 -
6.2 Briefly Describe the Function of SHELL-BASH And processing flow - 10 -
6.3 FORK process creation process of HELLO - 10 -
6.4 EXECVE process of HELLO - 10 -
6.5 Process execution of HELLO - 10 -
6.6 Exception and signal processing of HELLO - 10 -
6.7 Summary of this chapter - 10 -
Chapter 7 Chapter HELLO Storage Management - 11 -
7.1 HELLO Memory Address Space - 11 -
7.2 INTEL Logical Address to Linear Address Conversion - Segment Management - 11 -
7.3 HELLO Linear Address to Physical Address Conversion - Page Management - 11 -
7.4 Transformation from VA to PA supported by TLB and four-level page table - 11 -
7.5 Physical memory access supported by three-level CACHE - 11 -
7.6 Memory mapping when HELLO process is FORK - 11 -
7.7 Memory mapping when HELLO process is EXECVE - 11 -
7.8 Page fault and page fault interrupt handling - 11 -
7.9 Dynamic storage allocation management - 11 -
7.10 Summary of this chapter - 12 -
Chapter 8 IO management of HELLO - 13 -
8.1 IO device management method of LINUX - 13 -
8.2 Brief description of UNIX IO interface and its functions - 13 - 8.3
Implementation analysis of PRINTF - 13 -
8.4 Implementation analysis of GETCHAR - 13 -
8.5 This chapter Summary- 13
-Conclusion- 14
-Appendix- 15
-References- 16-
Chapter 1 Overview
1.1 Introduction to Hello
1. Create the .c file by writing the hello program with the editor gcc, and get the source program of hello.c.
2. Preprocess it through the C preprocessor (cpp) to generate the hello.i file (processing of include define).
3. Translate it with a C compiler (ccl) to generate the assembly language file hello.s.
4. Translate it into a relocatable object file hello.o through the assembler (as).
5. Run the linker program ld to combine hello.o and the system target file to create an executable target file hello. (As shown in the figure)
6. Enter ./shell through the shell, the shell creates a new process through the fork function, and then calls execve to map the virtual memory, and creates a space for the hello program through mmap.
7. The CPU fetches the code and data from the .text and .data sections in the virtual memory, and the scheduler plans a time slice for the process, and triggers an exception handling subroutine when there is an exception.
8. When the program finishes running, the parent process recycles the hello process and the child processes it created, and the kernel deletes the relevant data structures.
1.2 Environment and tools
1. Hardware environment: X64 CPU; 2.4GHz;
2. Software environment: Windows11 64-bit; Vmware 17; Ubuntu 17.1
3. Tools: codeblocks; gdb; Objdump; vs2022
1.3 Intermediate results
File name function
hello.c source Program
hello.i Preprocessed file
hello.s Compiled assembly file
hello.o Assembled relocatable target executable file
hello Linked executable file
hello.elf ELF format of hello.o
hello1.txt Disassembly of hello.o
hello2.txt Disassembly code
of hello hello1.elf hello ELF format
1.4 Summary of this chapter
This chapter generally introduces the "life" process of the hello program, as well as the basic information of the software and hardware environment, development and debugging tools during the experiment.
Chapter 2 Preprocessing
2.1 The concept and function of preprocessing
1. Preprocessing concept: The preprocessor (cpp) modifies the original C program according to the command beginning with the character #.
2. Preprocessing function: Modify the source code according to the preprocessing instructions in the source code. The preprocessing inserts the source code of the header file into the target file from the header file package of the system. The macros and constant identifiers have all been replaced by corresponding codes and values. Replace, and finally generate the .i file.
2.2 Preprocessing commands under Ubuntu
The command to preprocess the hello.c file in Linux is: gcc -E hello.c
2.3 Analysis of the preprocessing result of Hello
Using the command: gcc -E hello.c -o hello.i, the hello.i file can be generated, with a total of more than 3000 lines, which is still a readable C language program text file. The role of preprocessing is to expand the macros in the original program and include the contents of the header file into the file. For example, declare functions, define structures, define variables, define macros, etc. If there is a #define command in the code, the corresponding symbols will be replaced.
The header file library information involved in the hello.c file at the beginning of the hello.i file:
the end of the file is the main content of the hello.c file:
2.4 Summary of this chapter
This chapter introduces the related concepts and functions of preprocessing. After checking the hello.i file in practice, it supplements and replaces the source program. The preprocessor does perform a lot of expansion operations on the source code. After preprocessing The result is still a legal C language source file.
Chapter 3 Compilation
3.1 Concept and Function of Compilation
Compilation is the process of translating a program written in source language into a binary language recognizable by a computer. It includes five stages: lexical analysis, syntax analysis, semantic checking and intermediate code generation, code optimization and object code generation. The main function of the compiler is to enable the computer to understand the source language program, so as to write a language that the computer can run. Unlike interpreted languages, compilers do not need to translate the program line by line into the target language.
3.2 Commands compiled under Ubuntu
Enter the command gcc -S hello.i -o hello.s -fno-PIC -no-pie -m64 in the terminal to get the compilation result hello.s. As shown below:
3.3 Compilation result analysis of Hello
3.3.1 Initial part of assembly Section
name
function.file declares source file.text
code
section.section.rodata read-only data segment.globl
declares global
variable.type declares whether a symbol is a function type or a data
type.size Declare size.string
Declare a string.align
Declare how to align instruction or data storage address
3.3.2 Data
① String
There are two strings in the program, both of which are in the read-only data segment, as shown in the figure:
The array in hello.c is used as the second parameter of the main function, and each element of the array is a pointer to a character type. The starting address of the array is stored at -32 (%rbp) in the stack, as shown in the figure:
It is called twice to find the parameters and pass them to printf. These two strings are used as parameters of the printf function, as shown in the figure:
②Local variable i
The main function declares a local variable i, which will be placed on the stack when the compiler compiles. As shown in the figure, the local variable i is placed at -4 (%rbp) on the stack, as shown in the figure:
③Parameter argc
The parameter argc is used as the parameter passed by the user to main (the compiler will obtain the size of argc according to the parameters passed in by the user, that is, the number of passed parameters). It is also placed on the stack.
④Array: char *argv[]
char *argv[] is the second parameter of the main function, and each element of the array is a pointer to a character type (argv[0] generally indicates the executable file entered into the command line name). The starting address of the array is stored at position -32 (%rbp) in the stack, and is passed to printf by two calls.
⑤ Immediate number
The immediate number is directly reflected in the assembly code.
3.3.3 Global function
hello.c declares a global function int main(int argc, char *argv[]), the assembly code shows that the main function is a global function, as shown in the figure:
3.3.3 Assignment operation
The assignment operation in hello.c is i=0 in the for loop; it is realized by using the mov instruction in the assembly code, as shown in the figure:
The mov instruction is divided according to the byte size of the operand:
movb: one byte
movw: "word"
movl: "double word"
movq: "quad word"
3.3.4 Arithmetic operation
The arithmetic operation in hello.c is i++, assembly language addl $1, -4(%rbp), and other arithmetic operations are shown in the figure:
3.3.5 Relational operations ①
if(argc!=4); in hello.c is a conditional judgment statement. When compiling, this instruction is compiled as: cmpl $4,-20(%rbp), and the condition is set after the comparison code, and judge whether to jump according to the condition code. As shown in the picture:
② i<9 in hello.c is compiled as cmpl $8,-4(%rbp) as a judgment loop condition instruction, and the condition code is set to prepare for the next jle to use the condition code to jump. As shown in the picture:
3.3.6 Control transfer instructions
In assembly language, set the condition code first, and then perform control transfer according to the condition code. In hello.c, there are the following control transfer instructions:
① Determine whether argc is equal to 4. If argc is equal to 4, do not execute if statement, otherwise the if statement is executed, and the corresponding assembly code is:
②In the for loop, each time it is judged whether i is less than or equal to 8 to decide whether to continue the loop, the corresponding assembly code is:
3.3.7 Function operation
The following operations are performed when calling a function: (assuming that function A calls function B)
① Transfer control: when calling procedure A, the program counter (%rip) must be set to the starting address of function A code, and then return , the program counter (%rip) is set to the address of the instruction following the callA instruction.
② Passing data: Function A must be able to provide one or more parameters to function B, and B must be able to return a value to A.
③ Allocate and release memory: At the beginning, B may need to allocate space for local variables (the space for local variables is in the stack), and before returning, these spaces must be released.
Function operation in hello.c:
main function: parameter is int argc, char *argv[]
printf function: parameter is argv[1], argv[2]
exit function: parameter is 1
sleep function: parameter is atoi(argv[ 3])
getchar function: no parameters
3.4 Summary of this chapter
This chapter mainly introduces the basic process of the compiler processing the C language program, the function changes from the source code to the equivalent assembly code, and the compiler respectively converts the data from the C language, type conversion, assignment Statements, arithmetic operations, logic/bit operations, relational operations, control transfer and function operations are analyzed. By understanding the mechanism of these compiler compilations, we can easily translate assembly language into C language, which improves the anti- engineering capabilities.
Chapter 4 Assembly
4.1 The concept and function of assembly
Assembly language (Assembly Language) is any low-level language used in electronic computers, microprocessors, microcontrollers or other programmable devices, also known as symbolic language. It is a type of machine language in which operation codes are replaced by mnemonics, and address codes are replaced by address symbols or labels. In different devices, the assembly language corresponds to different machine language instruction sets, which are converted into machine instructions through the assembly process. There is a one-to-one correspondence between a specific assembly language and a specific machine language instruction set, so the assembly language may be different on different platforms.
The role of assembly language:
(1) Provide a low-level language closer to computer hardware for more efficient programs.
(2) Provide a more direct way for programmers to better understand how computers work.
(3) It can be used to implement complex software systems such as embedded systems, operating systems, and drivers.
(4) Assembly language can be used to write malware such as viruses and Trojan horses. 4.2 Command gcc hello.s -c -o hello.o
compiled under Ubuntu
4.3 Relocatable target elf format
Command: readelf -a hello.o > hello.elf
The content in the .elf file:
(1) ELF header:
The ELF header (ELF header) starts with a 16-byte sequence, which describes the word size and byte order of the system that generated the file. The rest of the ELF header contains information that helps the linker parse and interpret the object file, including the size of the ELF header, the type of object file (such as relocatable, executable, or shared), and the machine type (such as x86- 64), the file offset of the section header table, and the size and number of entries in the section header table. The location and size of the different sections are described by a section header table, where each section in the object file has a fixed-size entry. As shown in the picture:
(2) Section header:
record the name, type, address, offset, size, overall size, flag, link, information, and alignment of each section. As shown in the picture:
(3) Relocation section:
.rela.dyn stores the information that needs to be corrected in the .text section; any instructions that call external functions or reference global variables need to be corrected; instructions that call external functions need to be relocated; references to global Instructions for variables require relocation; instructions for calling local functions do not require relocation; there is no relocation information in the executable object file. The programs that need to be relocated are .L0 and .L1 in printf, puts, exit, sleepsecs, getchar, sleep and .rodata. As shown in the picture:
.rel.plt saves the information of the relocation table, and can use the lazy connection method. As shown in the picture:
(4) Symbol table:
.symtab, a symbol table, which stores information about functions and global variables defined and referenced in the program. Some programmers mistakenly believe that a program must be compiled with the -g option to obtain symbol table information . Virtually every relocatable object file has a symbol table in .symtab (unless the programmer deliberately removes it with the STRIP command). However, unlike the symbol table in a compiler, the .symtab symbol table does not contain entries for local variables. As shown in the picture:
4.4 Analysis of the result of Hello.o
Command: objdump -d -r hello.o > hello1.txt
Differences from hello.s:
(1) Branch transfer:
hello.s:
hello1.txt:
The disassembled jump instruction does not use a segment name such as .L3, but a definite address. In the disassembly code, the branch transfer is expressed as the function address + the offset in the segment. The operand of the jump instruction in the disassembly code does not use a segment name, because the segment name is just a mnemonic for easy writing in assembly language, so it obviously does not exist after assembly into machine language, but a definite address.
(2) The call to the function corresponds to the relocation entry
hello.s:
hello1.txt :
In the relocatable file, after call is no longer the specific name of the function, but a piece of information directed by the relocation entry. In the assembly file, it can be seen that the file name is directly added after the call.
(3) The immediate value becomes hello.s in hexadecimal format
:
hello1.txt :
In the relocatable object file, all immediate numbers are expressed in hexadecimal, because the conversion between hexadecimal and binary is more convenient than decimal, so they are all converted into hexadecimal.
4.5 Summary of this chapter
In this chapter, hello.s was assembled, and the hello.o relocatable object file was generated, and the ELF header, section header table, symbol table and relocatable section of the relocatable file were analyzed, and the hello The difference between .s and hello.o disassembly codes, and the correspondence between assembly language and machine language is analyzed.
Chapter 5 Link
5.1 The concept and function of link Link
(Link) is a mechanism for transferring parameters and control commands between various modules in an electronic computer program, and is a connection relationship between various modules in a computer program. During the running of the program, each module interacts and cooperates through links to complete specific tasks.
The concept of links can be applied in different fields. In the field of computer science, a link can refer to the connection relationship between program elements such as different files, data structures, and functions. In the field of the Internet, a link can refer to a connection relationship between web pages, and a web page can be jumped from one web page to another by clicking on a link. In the field of operating systems and network protocols, a link can refer to a connection relationship between a device and an application, such as a link in File Transfer Protocol (FTP).
The function of the link is to establish the link between the various modules, so that they can communicate with each other, share data and resources, and realize complex computing tasks. Through links, each module in the program can be modified and upgraded independently without affecting the operation of other modules. This makes program maintenance and updates much easier. At the same time, links also provide users with a more flexible and convenient way of use. Users can jump to pages, open files, perform operations, etc. by clicking on links.
5.2 Link command under Ubuntu
ld -o hello -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64- linux-gnu/crti.o hello.o /usr/lib/x86_64-linux-gnu/libc.so /usr/lib/x86_64-linux-gnu/crtn.o
5.3 The format of the executable target file hello
Command: readelf -a hello > hello1.elf
(1) ELF header: The difference between the hello file header and the hello.o file header is shown in the figure below. hello is an executable object file with 27 sections. As shown in the picture:
(2) Section header: declares all section information in hello, including size and offset. As shown in the picture:
(3) Relocation section.rela.text:
(4) Symbol table.symtab: store functions and global variables defined and referenced in the program.
5.4 The virtual address space of hello
The virtual address starts from 0x401000 and ends at 0x401ff0.
5.5 Analysis of link relocation process
Command: objdump -d -r hello > hello2.txt
and hello.o disassembled file comparison found that there are many sections in hello2.txt. There is only one .text section in hello1.txt, and there is only one main function, and the function address is also the default 0x000000. There are three sections.init, .plt, and .text in hello2.txt, and there are many functions in each section. The code of the library function has been linked into the program, each section of the program becomes more complete, and the address of the jump is also referenced. As shown in the figure:
it can be seen that the overall number of sections has increased, and the number of overall sections of the program has increased due to the loaded modules such as printf.o in the link. At the same time, the address of virtual memory is also slightly different. It is equivalent to resetting the memory address for it.
The process of relocation is divided into two steps:
(1) Relocation section and symbol definition: In this step, the linker merges all sections of the same type into a new aggregation section of the same type. For example, the .data sections from all input modules are all merged into one section, which becomes the .data section of the output executable object file.
(2) Symbol references in the relocation section: In this step, the linker modifies the references to each symbol in the code and data sections so that they point to the correct runtime address. To perform this step, the linker relies on relocatable entries, and those data analyzed in Section 5.3.
5.6 hello execution process
(1) start execution: _start, _libc_start_main
(2) execute main: _main, _printf, _exit, _sleep, _getchar
(3) exit: exit
program name program address
_start 0x400550
_libc_start_main 0x40057a
Main 0x400582
_printf 0x400500
_exit 0x
400530_sleep 0x400540_getchar
0x400510
5.7 Dynamic link analysis of Hello
The basic idea of dynamic link is to defer the linking process until runtime. When the program is running, the modules that need to be loaded will be dynamically linked. Therefore, dynamic linking can dynamically assemble the program at runtime without the need to determine the full path of all modules at compile time. When calling a shared library function, the compiler has no way to predict the runtime address of this function, because the shared library that defines it Modules can be loaded anywhere at runtime. The normal approach is to generate a relocation record for this reference, and then the dynamic linker resolves it when the program loads.
Delayed binding is implemented through GOT and PLT. According to the hello ELF file, the location of the GOT start table is 0x601000.
5.8 Summary of this chapter
This chapter mainly introduces the concept and function of links. Links can be divided into symbol definition and relocation, understand the ELF format of executable files, analyze the virtual address space of hello, relocation process, execution process, and dynamic link process , with a deeper understanding of links.
Chapter 6 hello process management
6.1 The concept and function of process
Process (Process) is a program in the computer about a running activity on a data set, it is the basis of system resource allocation and scheduling, and the basis of the operating system structure. In computer science, a process is a single execution of a program on a computer. It is an abstract concept used to describe the basic unit of CPU resource allocation. A process is a running activity of a program with independent functions on a certain data set. It applies for and owns system resources, and is a dynamic concept. A process is an active entity, not just the code of the program, but also the current activity, represented by the value of the program counter and the contents of the processing register.
The role of the process is:
(1) Improve resource utilization: The process is the basic unit for resource allocation and scheduling in the system. Through multiple processes executing multiple tasks concurrently, system resources can be fully utilized and resource utilization can be improved.
(2) Realize the separation of tasks: the process realizes the separation of the program, and different processes are independent of each other and will not interfere and affect each other, which helps to protect the stability and security of the system.
(3) Provide interactivity: the process can interact with the user through input and output operations, realize program debugging and testing, and provide a better user experience.
(3) Support parallel processing: The process supports concurrent execution of multiple tasks, which can realize parallel processing and improve the execution efficiency and response speed of the program.
In short, a process is an important concept of the operating system, the basic unit of the system for resource management and scheduling, and the basis for concurrent execution and parallel processing.
6.2 Brief description of the function and processing flow of Shell-bash
Shell (also known as Bash) is a command line interpreter used to execute commands and scripts on Unix, Linux and other operating systems. It is an important user interface in the operating system that allows the user to interact with the computer.
The role of Shell Shell:
(1) Provide an interface for the user to interact with the computer. Users can enter commands in the shell, which will be interpreted and executed by the shell.
(2) Execute the script. Shell can execute various script files, such as bash scripts, Python scripts, Perl scripts, etc.
(3) Management process. Shell can start, stop and manage processes in the operating system.
(4) Access the file system. Shell provides commands to access the file system, such as cd, ls, mkdir, etc.
Perform system administration tasks. Shell can perform various system management tasks, such as modifying environment variables, installing software, and so on.
Processing flow:
(1) Opening the Shell: The user opens the terminal and enters the Shell command line interface (also called the Shell command line interpreter).
(2) Prompt: Shell displays a prompt (also known as "$") to tell the user to enter the beginning of the command.
(3) User input command: the user inputs the command to be executed, such as cd, ls, mkdir, etc.
(4) Receive command: Shell receives the command entered by the user and saves it in memory.
(5) Parsing commands: Shell parses the commands entered by the user and breaks them down into smaller units, such as words, phrases, parameters, etc.
(6) Find commands: Shell searches for commands in the command path of the system.
(7) Execute the command: If the command is found, the Shell executes the command and outputs the result. If the command is not found, the shell displays an error message.
(8) Exit the Shell: When the user enters the exit command, the Shell exits and closes the terminal.
6.3 The fork process creation process of Hello
After entering the command line ./hello 2021112957 Wen Jiajie in the terminal, the shell first parses the command we entered. Since the command we entered is not a built-in shell command, the shell will call fork() to create A child process that is almost but not identical to the parent process. Through the fork function, the child process gets the same but independent copy of the user-level virtual address space of the parent process, with a different PID.
6.4 Hello's execve process
After calling the fork() function to create a child process, the child process calls the exceve function to load and run a new program in the context of the current child process, namely the hello program, which is called once and never returns.
When the execve() function is executed, all user data spaces of the current process will be cleared first, and the image of the new process will be loaded into this space. Then, copy the contents of the new argv and envp arrays into the new user data space. Finally, set the stack pointer of the new user data space as the stack pointer of the new program, and jump to the entry point of the new program to start the process execution of 6.5 Hello The key abstraction provided by the
process to the application program: an independent logical control flow , which provides the illusion that our program is using the processor exclusively. A private address space that provides the illusion that our program has exclusive use of the memory system. The process abstraction provided by the operating system: (1) Logical control flow: If we want to step through the program with a debugger, we will see a series of program counter (PC) values that uniquely correspond to the values contained in the program. Instructions in an executable object file, or in a shared object that is dynamically linked into a program at runtime. This sequence of PC values is called logical control flow, or simply logical flow. (2) Context switching: If the system call is blocked because it is waiting for an event to occur, the kernel can sleep the current process and switch to another process. The context is the state required by the kernel to restart a preempted process. Compare high-level exception control flow. (3) Time slice: Each time period in which a process executes part of its control flow is called a time slice. (4) User mode and kernel mode: The shell allows users to have the opportunity to modify the kernel, so some protective measures need to be set to protect the kernel, such as limiting the type of instructions and the scope of action. (5) Context information: The context is the state required by the kernel to restart a preempted process, which consists of general-purpose registers, floating-point registers, program counters, user stacks, status registers, kernel stacks, and various kernel data structures. Value composition.
Execution of the hello process: After the process calls the execve function, the process has allocated a new virtual address space for the hello program. Initially, hello runs in user mode and outputs hello 2021112957 Wen Jiajie, and then calls the sleep function. The process enters the kernel mode and runs the signal handler before returning to user mode. During the running process, the CPU continuously switches the context, so that the running process is divided into time slices, and the CPU is alternately occupied by other processes to realize process scheduling.
6.6 hello exception and signal processing
(1) exception can be divided into 4 categories: interrupt (interrupt), trap (trap), fault (fault) and termination (abort). The following are the attributes of these four categories of exceptions:
Interrupt: An exception caused by hardware or software, such as keyboard input, clock interrupt, etc.
Trap: A mechanism provided by the compiler or the operating system to catch certain types of exceptions.
Fault: An exception caused by a program error, such as division by zero, array out of bounds, etc.
abort: Thrown explicitly by the programmer to terminate program execution in an emergency.
All four types of exceptions can occur in a program and require the programmer to write appropriate code to handle them. Handling exceptions can improve the robustness and reliability of the program, so that the program can better cope with various abnormal conditions. ,
(2) Operation result
1. Normal operation
2. Press Ctrl-z
The default result of ctrl-z is to suspend the foreground job. The hello process is not recycled, but runs in the background. You can see with the ps command that the hello process has not been recycled. Call fg to bring it to the foreground. If the printing is finished, the process will be recycled, as shown in the figure below:
3. Press Ctrl+c
4. Non-stop random pressing
The irrelevant input is buffered to stdin, and is output to the result along with the printf command. As shown in the picture:
6.7 Summary of this chapter
(this chapter introduces the concept and function of process, the processing process and function of shell-bash, and focuses on the analysis of calling fork to create a new process, calling execve function to execute hello, the process execution process of hello, and hello encountered during runtime exception and signal handling.
Chapter 7 hello's storage management
7.1 hello's memory address space
1. Logical address: the address that appears in the assembly code after the program is compiled. A logical address is used to specify an operand or the address of an instruction. It is a segment identifier plus an offset specifying a relative address within the segment, expressed as segment identifier: offset within the segment.
2. Linear address: A step in the conversion process from logical address to physical address. The logical address is converted into a linear address through the segment mechanism, which is a combined form of descriptor: offset, and the linear address is used as input in the paging mechanism.
3. Virtual address: It is a linear address.
4. Physical address: The CPU finds the corresponding address of the real physical memory through the addressing of the address bus. The access of the CPU to the memory is accomplished through the front-side bus connecting the CPU and the north bridge chip. The memory addresses transmitted on the front side bus are all physical memory addresses.
7.2 Conversion of Intel Logical Address to Linear Address - Segment Management
Under the Intel platform, the format of the logical address is segment identifier: offset within the segment. A segment identifier consists of a 16-bit long field called a segment selector. The first 13 bits are an index number. The next 3 bits contain some hardware details. The segmentation mechanism converts a logical address into a linear address as follows:
- Determine the segment base address (Segment Base Address): Each segment has a corresponding segment base address, which is the starting address of the segment in memory.
- Calculate the offset (Offset): The offset in the logical address refers to the address relative to the base address of the current segment. The actual memory address can be calculated by the corresponding offset in the logical address and the segment base address.
- Add segment base address and offset: Add segment base address and offset to get linear address.
7.3 Conversion of Hello's linear address to physical address - paging management
Linear address, that is, the conversion between virtual address (VA) and physical address (PA) is completed through the paging mechanism.
The paging mechanism converts virtual addresses into physical addresses as follows: - Determine the page table start address (Page Table Start Address): The page table is a table used to convert virtual addresses to physical addresses. The operating system will load the starting address of the page table into memory.
- Determine the page table entry (Page Table Entry, PTE): In the page table, each entry is called a page table entry. Each page table entry contains a physical address and an access permission bit.
- Determine the Page Number in the virtual address: The virtual address consists of two parts: the page number and the offset within the page. The page number is used to look up the corresponding page table entry in the page table.
- Find the corresponding page table entry in the page table: Through the page number, the operating system can find the corresponding page table entry in the page table.
- Check that the page table entry contains a valid physical address: Before accessing a virtual address, the operating system must ensure that the virtual address corresponds to a valid physical address.
- Check the access permission bit of the page table entry: If the page table entry contains a valid physical address, the operating system also needs to check the access permission bit of the page table entry to determine whether it has permission to access the address.
- Add the physical address and offset: If the access permission bit allows access to the physical address, the operating system will add the physical address and the offset in the virtual address to get the actual physical address.
7.4 Transformation from VA to PA supported by TLB and four-level page table
- Determine the starting address of the page table: The operating system will load the starting address of the page table into memory.
- Determining the page number and offset within the virtual address: A virtual address consists of two parts: the page number and the offset within the page. The page number is used to look up the corresponding page table entry in the page table, and the page offset is used to calculate a part of the physical address after the corresponding page table entry is found.
- Find the corresponding page table entry in the page table: Through the page number, the operating system can find the corresponding page table entry in the page table.
- Check that the page table entry contains a valid physical address: Before accessing a virtual address, the operating system must ensure that the virtual address corresponds to a valid physical address.
- Check the access permission bit of the page table entry: If the page table entry contains a valid physical address, the operating system also needs to check the access permission bit of the page table entry to determine whether it has permission to access the address.
- Add the physical address and offset: If the access permission bit allows access to the physical address, the operating system will add the physical address and the offset in the virtual address to get the actual physical address.
- Check whether the TLB is hit: If the TLB (Translation Lookaside Buffer) caches the result of the virtual address to physical address translation just performed, the operating system can directly obtain the physical address from the TLB without performing another virtual address to physical address translation.
- Repeat steps 1-7 until the final physical address is found.
7.5 Physical memory access supported by L3 Cache
L3 cache (L3 cache) is the cache level of computer CPU. Typically, the CPU's cache is divided into four levels: L1, L2, L3, and L4. Among them, L1 and L2 caches are usually integrated in the CPU, while L3 caches are located on the motherboard, and L4 is a higher-speed storage device, such as SSD. When the CPU accesses memory, it needs to follow a certain access sequence and mechanism. The following are the physical memory access steps supported by the three-level Cache: - Determine whether the data that needs to be accessed is in a register of the CPU. If the data is already in a register, the CPU can directly access that data.
- If the data is not in a register, the CPU first checks the level 1 cache (L1 cache). If the data is in L1 cache, the CPU can directly access that data.
- If the data is not in the L1 cache, the CPU checks the L2 cache (L2 cache). If the data is in L2 cache, the CPU can directly access that data.
- If the data is not in the L2 cache, the CPU checks the L3 cache (L3 cache). If the data is in L3 cache, the CPU can directly access that data.
- If the data is neither in the L1 cache nor in the L2 cache and L3 cache, the CPU needs to read the data from the main memory (such as RAM). Reading data from main memory is much slower than reading data from cache.
- In order to improve the efficiency of CPU accessing data, the operating system and hardware will manage the data in the cache according to factors such as data usage frequency and importance. If data is no longer being used frequently at some point, it may be evicted from the cache to make room for new data.
7.6 Memory mapping when hello process forks
When the program calls the fork() function, it returns twice: once in the parent process and once in the child process. This is achieved by using the copy-on-write technique, which allows parent and child processes to share the same virtual address space.
After the fork() call, the virtual address spaces of the parent and child processes are mapped to the same physical memory, but they are independent of each other. That means they can access the same virtual address at the same time, but they won't interfere with each other. This is because each process has its own virtual address space, and the OS makes sure they don't conflict with each other.
In Linux, the memory mapping relationship between the parent process and the child process is represented by the copy control area (VMA, Virtual Memory Area). Each VMA object represents a range of virtual addresses within a process. The VMA object contains information about the access rights for that address range, mapped physical memory pages, and so on.
After the fork() call, the VMA objects of the parent and child processes are copied to reflect their respective virtual address spaces. However, the actual physical memory pages are only copied when they are accessed. This is because the fork() function uses Copy-on-Write technology, which allows parent and child processes to share the same physical memory pages until one of them tries to modify those pages.
7.7 Memory mapping during hello process execve
In Linux system, the execve() function is used to load the new process image into the address space of the existing process, which implements the file sharing and replacement mechanism, and has more flexible and efficient features .
When the execve() function is called, the following operations are performed: - Open a file: call the open() function to obtain the file data structure of the specified file path.
- Copy parameters: Copy the command line parameters and environment variables of the user address space to the kernel address space.
- Release memory: release memory related to the original program, such as open files, shared libraries, etc.
- Modify page tables: Update the process's page tables to reflect the new process's address space layout.
- Executing code: After modifying the page table, the CPU starts executing the code of the new process.
7.8 Page fault and page fault interrupt processing
Page fault (Page Fault) refers to the situation that the page (or page) that needs to be accessed is not in the main memory (RAM) when the computer executes the program. A page fault occurs when a program accesses a page that is not in main memory.
Page fault interrupt processing refers to the process in which the operating system intervenes and handles the fault when a page fault occurs. The following are the basic steps of page fault handling: - Generate an interrupt: When a program accesses a page that is not in the main memory, the CPU generates a page fault interrupt signal to transfer control to the operating system.
- Interrupt handler execution: The operating system will save the context of the current program (including register status, program counter, etc.), and start executing the page fault interrupt handler.
- Determine the cause of the page fault: The page fault interrupt handler will first determine the cause of the page fault, which may be because the page does not exist in the main memory (page fault), or the page exists in the main memory but has no legal access rights (protection fault) .
- Handle page faults: If it is a page fault, the operating system needs to load the required page from the auxiliary storage (usually the hard disk) into the main memory, update the data structure such as the page table, and make the page visible to the program.
- Handling protection faults: In case of protection faults, the operating system checks whether the program has sufficient permissions to access the page. If the privileges are insufficient, the operating system terminates the execution of the program or grants appropriate privileges.
- Restoring context: After handling page faults or protection faults, the operating system will restore the context of the interrupted program, including register status, program counter, etc.
- Re-execute the interrupted instruction: The operating system returns control to the interrupted program and re-executes the instruction that caused the page fault interrupt.
7.9 Dynamic storage allocation management
The basic methods and strategies of dynamic storage allocation management are as follows: - First Fit: Arrange the free area according to the starting address from small to large, search from the beginning of the chain until a free area that meets the requirements is found, and a memory space equal to the requested size is drawn from it.
- Cycle First Adaptation (Next Fit): When searching for a free area, instead of searching from the head of the chain every time, it starts searching from the next free area of the free area found last time until it finds a free area that meets the requirements So far, and draw a memory space equal to the requested size from it.
- Best Fit: Always allocate the smallest free area that meets the requirements to the requesting job, that is, in the free area table, arrange the free area from small to large, build an index, and when the user job requests memory When there is space, find the first free area that satisfies the job from the index table and give it to it.
- Worst Fit: Always assign the largest free area to the requesting job, and the free partitions in the free area table (free area chain) should be sorted by size from large to small, starting from the header to find the first A free partition that meets the requirements is allocated to the job.
- Available space table (Free List): directory table and linked list.
- Boundary Tagging: There are marks on the head and bottom boundaries of each memory area to identify whether the area is an occupied block or a free block.
The following is the general process of dynamically applying for memory:
1. Determine memory requirements: First, the program needs to determine the required memory size. This can be determined by calculating the size of the data structure, the amount of data that needs to be stored, etc.
2. Call the memory allocation function: In general, the programming language or operating system provides a memory allocation function, such as malloc() in C language or new operator in C++. The program can call these functions to apply for the required memory space.
3. Specify the memory size: When calling the memory allocation function, you need to specify the size of the required memory. The function tries to find a block of memory of sufficient size and allocate it to the program.
4. Allocate memory space: The memory allocation function will search for a continuous space of sufficient size in the available memory area. If enough space is found, it is marked as allocated and a pointer to the block of memory is returned.
5. Error handling: If there is not enough continuous memory space for allocation, the memory allocation function may return a null pointer or throw an exception, indicating that the memory allocation failed. Programs can perform error handling based on the return value, such as freeing allocated memory and take appropriate actions.
6. Using memory: Once the memory is allocated successfully and a pointer to the memory block is returned, the program can use the memory space to store data or perform other operations.
7. Release memory: After dynamically applying for memory, when the memory is no longer needed, the memory should be released explicitly. This can be done by calling a memory deallocation function, such as free() in C or the delete operator in C++. After the memory is freed, the memory space can be reallocated to other programs that need it.
7.10 Chapter Summary
This chapter mainly introduces hello's memory address space, intel's segment management, and hello's page management. Using intel Core7 in a specified environment, it introduces the conversion of virtual address VA to physical address PA, physical memory access, and analyzes the hello process fork memory mapping during hello process execve, page fault and page fault interrupt processing and dynamic storage allocation management.
Chapter 8 hello's IO management
8.1 Linux IO device management method
(arranged in the following format by yourself, delete when editing)
Device modeling: file
device management: unix io interface
8.2 Brief introduction to Unix IO interface and its functions
(arranged in the following format by yourself , delete when editing)
8.3 Analysis of the implementation of printf
(the following format is self-arranged and deleted when editing)
https://www.cnblogs.com/pianist/p/3315801.html
Generate display information from vsprintf, to write system function, to trap -System call int 0x80 or syscall, etc.
Character display driver subroutine: from ASCII to font library to display vram (storing RGB color information of each point).
The display chip reads the vram line by line according to the refresh rate, and transmits each point (RGB component) to the liquid crystal display through the signal line.
8.4 Analysis of the implementation of getchar
(arranged in the following format, delete when editing)
asynchronous exception - keyboard interrupt processing: keyboard interrupt processing subroutine. The accepted key scan code is converted into ascii code and saved to the keyboard buffer of the system.
Getchar calls the read system function, reads the ascii code of the key through the system call, and does not return until the Enter key is received.
8.5 Summary of this chapter
(The following format is self-arranged and deleted during editing)
(Chapter 8 1 point)
Conclusion
The process that Hello went through is as follows:
- Input: Input the source code to get the source program of hello.c.
- Preprocessing: Preprocessing with the command gcc -E makes hello.c grow into hello.i.
- Compile: Compile with the command gcc -S to make hello.i grow into hello.s (assembly language file).
- Assembly: Compilation using the command gcc -c makes hello.s grow into hello.o (relocatable object file)
- Link: link hello.o with the relocatable object file and the dynamic link library to become the executable object program
hello, and the executable hello program is officially born. - Run: Enter ./hello in the shell 2021112957 Wen Jiajie 1
- Create a subprocess: Since the terminal input is not a built-in shell command, the shell calls the fork () function to create a subprocess.
- Loading program: the shell calls the execve function, starts the loader, maps the virtual memory, and after entering the program entry, the program starts to load the physical memory, and then enters the main function.
- Executing instructions: The CPU allocates a time slice for the process. In a time slice, hello enjoys CPU resources and executes its own control logic flow sequentially.
- Access memory: The MMU maps the virtual memory address used in the program into a physical address through the page table.
- Dynamically apply for memory: printf will call malloc to apply for memory in the heap from the dynamic memory allocator.
- Signal management: When the program is running, we enter Ctrl+c, the kernel will send a SIGINT signal to the process and terminate the foreground job. When Ctrl+z is entered, the kernel will send a SIGTSTP signal to the process and stop the foreground job from being suspended.
- Termination: When the execution of the child process is completed, the kernel arranges for the parent process to recycle the child process and pass the exit status of the child process to the parent process. The kernel deletes all data structures created for this process. At this point, hello's life is over!
Attachment
hello.c Source program
hello.i Preprocessed file
hello.s Compiled assembly file
hello.o Compiled relocatable target execution file
hello Linked executable file
hello.elf ELF format of hello.o
hello1 .txt hello.o disassembly
hello2.txt hello disassembly
hello1.elf hello ELF format
References
[1] In-depth understanding of the original computer system book 3rd edition - text version.pdf
[2] https://blog.csdn.net/sy06106/article/details/118274118