Article directory
key words:
Physical address : The address seen by the memory unit, the address of the main memory in the machine, including RAM and ROM
Logical address : The address generated by the CPU, the address used by the program running in the memory, also known as the virtual address
Linear address:
Logical address: 0～Max, physical address: R～R+Max
Operation is the mapping from virtual address to logical address , which is completed by the memory unit management MMU device.
Segmentation mechanism
paging mechanism
MMU
User space
Kernel space
introduce
Memory allocation strategy, continuous memory allocation is one of the simplest methods, dividing the memory into multiple fixed-size partitions, each partition can only accommodate one process. There are three commonly used strategies for selecting memory blocks that meet the requirements of the process: 1. First adaptation (the first block meets the requirements) 2. Best adaptation (the smallest one is large enough) 3. Worst adaptation (the largest one) first
adaptation Both the best fit and the best fit will have external fragmentation problems . The solution is: compaction, moving the memory contents, and merging all free space into a single block.
Allow non-contiguous physical address space , common ones include paging and segmentation
MMU
Modern operating systems generally use a virtual memory management (Virtual Memory Management) mechanism, which requires support from the MMU (Memory Management Unit) in the processor. First, two concepts are
introduced, virtual address and physical address . If the processor does not have an MMU, or has an MMU but is not enabled, the memory address sent by the CPU execution unit will be directly transmitted to the chip pin and received by the memory chip (hereinafter referred to as physical memory to distinguish it from virtual memory). This is called Physical Address (hereinafter referred to as PA)
physical address
The CPU accesses units in the memory through addresses. The addresses are divided into virtual addresses and physical addresses. If the CPU does not have an MMU (Memory Management Unit), or it has an MMU but is not enabled, the CPU core will fetch instructions or access the memory. The sent address will be directly transmitted to the external address pin of the CPU chip and directly received by the memory chip (hereinafter referred to as physical memory to distinguish it from virtual memory). This is called the physical address (Physical Address, hereafter referred to as PA)
. Memory chip-level unit addressing corresponds to the address bus connecting the processor and CPU.
linear address
If the CPU enables MMU, the address sent by the CPU core will be intercepted by the MMU. The address from the CPU to the MMU is called a virtual address (Virtual Address, hereinafter referred to as VA), and the MMU translates this address into another address and sends it to the CPU chip. On the external address pin, that is, the virtual address is mapped to the physical address.
The MMU maps the virtual address to the physical address in units of pages. For 32-bit CPUs, one page is usually 4K. For example, virtual address 0xb700 1000 ~ 0xb700 1fff is a page, which may be mapped to physical address 0x2000 ~ 0x2fff by the MMU. A physical page in physical memory is also called a page frame (Page Frame). The kernel cannot directly access the physical address. However
, Because the difference between the virtual address and the physical address of the kernel is only a difference of 0xc0000000, it is easy to find the virtual address from the physical address or the physical address from the virtual address. ± This difference is enough.
Virtual Memory
This is an abstract description of the entire memory (do not match the number inserted on the machine). It is relative to physical memory and can be directly understood as "unreal" and "fake" memory. For example, a memory address of 0x08000000 is not the address 0x08000000 - 1 in the large array of physical addresses. element;
the reason for this is that modern operating systems provide an abstraction of memory management, that is, virtual memory. The process uses the address in virtual memory, and the operating system assists the relevant hardware to "convert" it into a real physical address. This "conversion" is the key to all discussions.
With such an abstraction, a program can use an address space that is much larger than the real physical address. (Tear down the east wall and mend the west wall, this is how banks do it), even multiple processes can use the same address. Not surprising, since the translated physical addresses are not the same.
——You can decompile the connected program and find that the linker has assigned an address to the program. For example, to call a certain function A, the code is not call A, but call 0x0811111111. In other words, the address of function A The address has been decided. Without such a "conversion", without the concept of a virtual address, this simply won't work
logical address
For the sake of compatibility, Intel has retained the segmented memory management method from ancient times. The logical address refers to the address of an operand or an instruction in a machine language instruction. In the above example, the address 0x08111111 assigned by the connector to A is the logical address.
——But I'm sorry, saying this seems to violate the requirements for logical addresses in Intel's mid-segment management. "A logical address is composed of a segment identifier plus an offset of the relative address within the specified segment, expressed as [Segment identifier: intra-segment offset], that is to say, the 0x08111111 in the above example should be expressed as [A's code segment identifier: 0x08111111], so that it is more complete."
To convert an address in a virtual memory space into a physical address, the CPU needs to perform two steps: first, a logical address (actually an intra-segment offset) is given. The CPU uses its segment memory management unit. The logical address is converted into a thread address, and then converted into the final physical address using its page memory management unit.
Doing this two conversions is indeed very troublesome and unnecessary, because the linear address can be directly abstracted to the process. The reason for this redundancy is entirely for compatibility.
Relative to the address used by the programmer, or that the program does not need to know the specific actual address management number, it only needs to be used within the allowable range of the system (operation) (at this time, an address under the control of an algorithm is used. In fact, It is just a program operation mode generated by borrowing the concept of address), and what it describes is convenience, that is, a linear (best) arrangement of programs (instructions). Its size is generally determined by the operating system
Memory management
Memory and addressing
Memory is an array of ordered bytes. Each byte has a unique memory address.
Memory addressing refers to locating the address of a specified data object stored in memory. A data object refers to a data object stored in memory. A numerical value or string of a specified data type
- Usually memory addresses start from 0. For 80X86, the address bus width is 32 bits, so there are 2^32 different physical addresses in total, that is, the memory physical address space is 4G
- For 80X86, an instruction mainly consists of the Opcode operation code and the Oprand operation object, that is, the operand. The operand can be located in a register or in memory. To locate the operand in the memory, memory addressing is required. For There are many different addressing schemes depending on the data type of the addressing object. In order to perform memory addressing,
X86 provides 6 registers for the segment part to store segment selectors : CS, DS, ES, SS, FS. Among them, CS is used for Addressing the code section, while the combat section exclusively uses the SS registerIn order to specify the offset address within the segment of the memory operand, X86 specifies many ways to calculate the offset, called the instruction addressing mode: offset address = base address + (index x scale factor) + offset displacement
- X86 uses a **Segment segment** addressing technology, which divides the memory space into linear areas of one or more segments. Therefore, addressing a data object in the memory requires the use of the **start of a segment. It consists of two parts: address (segment address)** and an offset address within the segment.
- The segment address part is specified using a 16-bit segment selector , of which 14 bits can select 2^14 (16384) segments.
- The offset address within a segment can be 0~4G, that is, a segment can be up to 4G.
- In the program, a 48-bit address or long pointer composed of a 16-bit segment and a 32-bit offset is called a logical address (virtual address) . It uniquely determines the segment address and intra-segment offset address of a data object , and only consists of The 32-bit offset address or address specified by the pointer is based on the object address of the current segment.
address translation
Any complete memory management system contains two key parts: protection and address translation
- Protection measures are provided to prevent one task from accessing memory areas of another task or operation.
- Address translation allows the operating system to be flexible when allocating memory to tasks, and because we can prevent certain physical addresses from being mapped by any logical addresses, the address translation process also provides a memory protection function. Physical memory in computers
is A linear array of bytes, each byte has a unique physical address: the address in the program is a logical address composed of two parts. This logical address cannot directly access the physical address, but needs to use an address transformation mechanism to transform or Mapping to a physical memory address, the memory management mechanism is used to convert this logical address into a physical memory address.
In order to reduce the information required to determine the address conversion, conversion or mapping usually uses memory blocks as the unit of operation. Segmentation mechanism and paging Mechanisms are two widely used address translation techniques. They differ in how logical addresses are organized into blocks of mapped memory, what translation information is specified and how programmers operate it. Both segmentation and paging use resident memory. tables to specify their respective transformation information, these tables can only be accessed by the operating system
address translation
segmentation mechanism
Convert logical addresses to linear addresses
The segmentation mechanism in the figure above divides the address space addressable by the processor into smaller protected address space regions called segments. Use can be used to store program code, data and stack, or system data structures. When running multiple programs, assign each program its own segment to ensure that the programs do not interfere with each other. You can also classify segments and assign different permissions to different types of segments.
All segments used in the system are contained in the processor's linear address space. When a certain byte in a segment needs to be located (this is addressing), the program must provide a logical address. The logical address contains a segment selector and an offset.
The segment selector is unique in the system. The segment selector provides the offset of the segment descriptor. The segment descriptor is a data structure in the segment descriptor table. Each segment has a segment descriptor, which specifies the size of the segment, access permissions, privileged machine of the segment, segment type and base address of the segment (the position of the first byte of the segment in the star address space, also called segment address)
The offset of the logical address (that is, the offset address) plus the segment address can be used to locate a certain byte in the segment. Therefore, the segment address + offset address is the address in the processor's linear address space (linear address).
Linear addresses and physical addresses have the same data structure. For a two-dimensional logical address space (why two dimensions? Do segment addresses and offset addresses each count as one dimension?), they are both one-dimensional.
The virtual address (logical address) space can only have a maximum of 16k segments (2^14= 16348 segments), which is divided into two parts. The first half is the global address space, and the second half is the local address space. Each segment is up to 4GB. In this way, the maximum virtual address space can reach 64TB (2^14 ∗ 2^32 = 2^46). Both linear address space and physical address space are 4GB. If paging is disabled, the linear address space is the physical address space.
Paging mechanism
To convert linear addresses into physical addresses , the processor paging mechanism divides the linear address space into pages, and then these linear address space pages are mapped to pages in the physical address space. Multitasking systems usually
define a linear address space that is larger than the actual physical memory. Much larger, so some way of "virtualizing" the linear address space needs to be used, namely virtual storage technology. Virtual storage is a memory management technology that allows programmers to create the illusion that the memory space is much larger than the actual physical memory.
The paging mechanism supports virtual storage technology. Use a small block of physical memory (RAM or ROM) and some external storage space (such as a hard disk) to simulate a large linear address. Divide each segment into pages of the same size (usually 4KB per page). The operating system maintains a page directory and some page tables. When a program wants to access a certain location in the linear address space, the processor will use the page to access a certain location in the linear address space. Directories and page tables convert the linear address into a physical address, and then reads and writes can be performed on that memory location. If the page currently being accessed is not in physical memory, the processor will generate a page fault exception to interrupt the execution of the program. Then the operating system can read the page from the hard disk into physical memory and continue executing the interrupted program. . If the operating system implements paging strictly, the exchange of pages between physical memory and the hard disk is transparent to correctly executing programs.
The paging mechanism uses fixed-size memory blocks, while the segmentation mechanism uses variable-size memory. Whether in physical memory or on the hard disk, paging uses fixed-size blocks more suitable for physical memory, while the segmentation mechanism uses variable-size blocks. Variable blocks are more suitable for processing logical partitions of complex systems. Logical block sizes can be defined to fit memory units without being limited by fixed-size pages. Each segment can be processed as a unit, simplifying segment protection and sharing. .
enable
The paging mechanism can be enabled by setting PG in control register CR0.
If PG=1, paging operation is enabled, and the processor will use the paging mechanism to convert linear addresses into physical addresses.
If PG=0, the paging mechanism is disabled, and the linear address generated by the segmentation mechanism is directly used as the physical address.
The paging mechanism implements the process of converting linear addresses to physical addresses
80x86 uses 4K byte fixed-size pages, each page is 4KB and is aligned on 4K address boundaries.
This means that the paging mechanism divides the 2^32 bytes (4GB) linear address space into **2^20** (1M = 1048576) pages. The paging mechanism operates by relocating pages in the linear address space into the physical address space.
Since a 4K page is mapped as a unit and is on a 4K boundary, the lower 12 bits of the linear address can be used as an intra-page offset directly as the lower 12 bits of the physical address . The redirection function performed by the paging mechanism can be regarded as converting the high 20 bits of the linear address to the high 20 bits of the corresponding physical address .
The linear-to-physical address translation functionality has been extended to allow a linear address to be marked as invalid, rather than having it generate a physical address. A page can be marked as invalid in the following two situations:
- Linear address not supported by operating system
- The corresponding virtual memory page in the system is on disk rather than in physical memory. In the first case, the program that generated the invalid address must be terminated. In the second case, the invalid address is actually a request for operating system virtual memory. The manager
loads the corresponding page from disk into physical memory for program access. Because invalid pages are often associated with virtual storage systems, they are called non-present pages, determined by an attribute in the page table called present.
When using paging, the processor divides the linear address space into fixed-size pages (4KB). These pages can be mapped into physical memory or disk storage space. When a program references a logical address in memory, the processor Convert the logical address into a linear address, and then use the paging mechanism to convert the linear address into the corresponding physical address.
If the page containing the linear address is not currently in physical memory, the processor generates a page fault exception. The page fault exception handler will ask the operating system to load the corresponding page from the disk into physical memory (different pages in physical memory may be written to the disk during the operation). After the page is loaded into physical memory, returning from the exception handler causes the exception instruction to be re-executed. The information used by the processor to translate linear addresses into physical addresses and to generate page faults is contained in page directories and page tables in storage and memory.
The difference between paging mechanism and segmentation mechanism
The biggest difference between paging and segmentation is that paging uses fixed-length pages.
The length of a segment is usually the same as the code or data structure stored in it. Unlike paragraphs, pages have a fixed length.
If only segmented address translation is used, a data structure stored in physical memory will contain all of its parts. If paging is used, part of a data structure can be stored in physical memory and part on disk.
To reduce the number of bus cycles required for address translation, recently accessed page directories and page tables are stored in a processor buffer device called the Translation Lookup Buffer (TLB). The TLB can satisfy most requests to read page directories and page tables without using bus cycles. Extra bus cycles are used to read the page table entry from memory only if the TLB does not contain the required page table entry. This usually occurs when a page table entry has not been accessed for a long time.
Page table structure
The paging translation function is described by a memory-resident table called a page table, which is stored in the physical address space.
The page table can be thought of as a simple 2^20 array of physical addresses. The linear to physical address mapping function can be viewed simply as performing an array lookup. The upper 20 bits of the linear address constitute the index value of this array, which is used to select the physical (base) address of the corresponding page.
The lower 12 bits of the linear address give the offset in the page, which together with the page's base address ultimately forms the corresponding physical address. Since the page base address is aligned on a 4K boundary, the lower 12 bits of the page base address must be 0, which means that the upper 20 bits of the page base address and the 12-bit offset address can be combined together to get the corresponding physical address.
The size of each page table entry in the page table is 32 bits. Since only 20 bits are needed to store the physical base address of the page, the remaining 12 bits can be used to store attribute information such as whether the page exists. If the page table of the linear address index is marked as existing, it means that the entry is valid and we can obtain the physical address of the page from it. If the item indicates that it does not exist, an exception will be generated when accessing the corresponding physical interface.
Two-level page table structure
The page table contains 2^20 entries, and each entry occupies 4 bytes. If stored as a table, it will occupy up to 4M of memory. To reduce memory usage, 80x86 uses a two-level page table. The conversion of the upper 20 bits of the linear address to the physical address is performed in two steps, with 10 bits converted each time.- The first-level table is called the page directory , which is stored in a 4K page and has 2^10 entries of 4 bytes in length. These representations point to corresponding secondary tables. The top 10 bits (31-22) of the linear address are used as indexes into the table .
- The second level is called the page table , which is also a 4K page in length and has at most 1K 4-byte entries. Each 4-byte entry contains the 20-bit physical base address of the associated page. The secondary page table uses the middle 10 bits (21-12) of the linear address as the entry index value to obtain the entry containing the base address of the physical address of page 20. The 20-bit page physical base address and the lower 12 bits (intra-page offset) of the linear address are combined to obtain the output value of the paging conversion process, which is the corresponding final physical address.
The search process of the secondary page table. The CR3 register specifies the base address of the page directory table. The upper 10 bits of the linear address are used to index this page directory table to obtain a pointer to the associated second-level page table. The middle 10 bits of the linear address space are used to index the secondary page table to obtain the upper 20 bits of the physical address. The lower 12 bits of the linear address are directly used as the lower 12 bits of the physical address, thus forming a complete 32-bit physical address.
non-existent page table
The secondary page table structure allows the page table to be dispersed across various pages of memory without needing to be stored in contiguous 4MB memory blocks. Additionally, there is no need to allocate secondary page tables for non-existent or unused portions of the linear address space. Although directory table pages must reside in memory, secondary page tables can be reallocated when needed. This makes the size of the page table structure correspond to the actual size of the linear address space used.
Each entry in the page directory table also has an existence attribute, similar to the entry in the page table. The existence attribute in the page directory entry indicates whether the corresponding secondary page table exists. If the secondary page table specified by the directory table exists, then the second step of the table lookup process will proceed normally by accessing the secondary table. If the existence bit indicates that the corresponding secondary table does not exist, the processor will generate an exception to notify the operating system. The presence attribute in the page directory entry allows the operating system to allocate secondary page table pages based on the actual linear address range used.
The presence bit in the directory entry can also be used to store secondary page tables in virtual memory. This means that only part of the secondary page table needs to be in physical memory at any time, while the rest can be kept on disk. Page directory entries corresponding to page tables in physical memory will be marked as present to indicate that they are available for paging translation. The page directory entry corresponding to the page table on disk will be marked as non-existent. Exceptions caused by the non-existence of the secondary page table will notify the operating system to load the missing page table from the disk into physical memory. Storing page tables in virtual memory reduces the amount of physical memory required to hold paging translations.
Page table entry format
The entry format of the page directory and page table is as shown in the figure below. Bits 32-12 contain the high 20 bits of the physical address, which are used to locate the physical base address of a page (also called a page frame) in the physical address space. The lower 12 bits of the entry contain page attribute information.
P bit 0 is a presence flag, used to indicate whether the entry is valid for address translation. P=1 means valid; P=0 means invalid. During the page conversion process, if the entry in the page directory or page table involved is invalid, an exception will be caused. If P=0, then except that the table entry is invalid, the remaining bits can be used freely by the program, as shown in b in the figure.
R/W bit 1 is the read/write flag. If equal to 1, the page can be read, written, or executed. If 0, the page is read-only or executable. When processing is running at superuser privilege level (level 0, 1, or 2), the R/W bit has no effect. The R/W bit in a page directory entry affects all pages to which it is mapped.
U/S bit 2 is the user/superuser flag. If it is 1, then programs running at any privilege level can access the page. If it is 0, then the page can only be accessed by programs running at the super privilege level. The U/S bit in a page directory entry affects all pages to which it is mapped.
Bit 5 of A is the accessed flag. When the processor accesses the page mapped by the page table entry, this flag of the page table entry is set to 1. When the processor accesses any page mapped by the page directory entry, this flag of the page table entry is set. is 1. The processor is only responsible for setting this flag, and the operating system can reset this flag periodically to count page usage.
D bit 6 is a flag that the page has been modified. When the processor performs a write operation on a page, it sets the D flag of the corresponding page table entry. The processor does not modify the D flag in the page directory entry.
AVL This field is reserved for program use only. The processor will not modify these bits, nor will future processor upgrades.
virtual storage
The presence flag P in the page directory and page table entries provides necessary support for virtual storage using paging technology. If the page in the linear address space exists in physical memory, the flag P=1 in the corresponding entry, and the entry contains the corresponding physical address. Table entries whose pages are not in physical memory have flag P=0. If a program accesses a page that does not exist in physical memory, the processor generates a page fault exception. At this time, the operating system can use this exception handling process to transfer the missing page from the disk into physical memory, and store the corresponding physical address in the table entry. Finally, the flag P= is set before returning to the program to re-execute the instruction that caused the exception. 1.
The accessed flag A and the modified flag D can be used to effectively implement virtual storage technology. By periodically checking and resetting all A flags, the operating system can determine which pages have not been accessed recently and can become candidates for moving to disk. Assume that when a page is read into memory from the disk, its dirty flag D=0, then when the page is moved out to the disk again, if the D flag is still 0, the page does not need to be written to the disk (because it has not been change). If D=1 at this time, it means that the page has been modified, so the page must be written to the disk.
Inter-task protection and privilege-level protection
- Protection between tasks: Give each task a different logical address space to completely isolate each task. This is achieved by mapping the logical address of each task to a different physical address.
Place each task in a different virtual address space, and set a different mapping method from logical address to physical address for each task. The logical address of one task is mapped to a part of the physical memory, and the logical address of another task is mapped to a part of the physical memory. The address is mapped to another area of physical memory. The logical address of a task will not be mapped to the physical memory area mapped by the logical address of other tasks, so all tasks are isolated. Just give each task its own independent mapping table, and each task will have a different address translation function. In 80X86, each task has its own segment table and page table. When the processor switches to execute a new task, the key to task switching is to switch to the transformation table of the new task.
By arranging the same virtual address to physical address mapping part in all tasks, and storing the operating system in this common virtual address space part, the operating system can be shared by all tasks. This portion of the virtual address space that is shared by all tasks is called the global address space. This is also the part of virtual address space used by modern Linux operating systems.
The unique virtual address space of each task is called the local address space. The local address space contains private code and data that needs to be distinguished from other tasks in the system. Since each task has a different local address space, a reference at the same virtual address from two different tasks will be translated to a different physical address. This is how the operating system can give each task the same virtual address and still isolate each task. On the other hand, references by all tasks to the same virtual address in the global address space will be translated to the same physical address, which provides support for the sharing of common code and data (such as operating systems).
- Privilege level protection
In a task, 4 execution privilege levels are defined, which are used to restrict access to each section of the task based on the sensitivity of the data contained in the section and the trustworthiness of different program parts in the task. The most sensitive data is given the highest privilege level, and they can only be accessed by the most trusted parts of the mission. Less sensitive data is assigned a lower privilege level and can be accessed by less privileged code within the task.
Privilege levels are represented by numbers 0 through 3, with 0 having the highest privilege level and 3 being the lowest. Each memory segment is associated with a privilege level. This privilege level restricts access to this segment to only programs with sufficient privilege level. The processor obtains and executes instructions from the segment specified by the CS register. The current privilege level, or CPL, is the privilege level of the currently active code segment. It defines the privilege level of the currently executed program and determines which segments can be accessed by the program.
Whenever a program attempts to access a segment, the current privilege level is compared to the segment's privilege level to determine whether access is granted. Programs executing at a given CPL level can access data segments of the same level or lower levels. Any reference to a higher-level segment is illegal and an exception is thrown to notify the operating system. Each privilege level has its own program stack to avoid the protection issues caused by using a shared stack. When a program switches execution from one privilege level to another, the stack segment also changes to the new level of stack.