A physical memory and virtual memory
We know that reading and writing data directly from physical memory is much faster than reading and writing data from hard disk. Therefore, we hope that all data reading and writing are completed in memory, and memory is limited, which leads to physical The concept of memory and virtual memory.
Physical memory is the memory size provided by the system hardware. It is real memory. Compared with physical memory, there is also a concept of virtual memory under Linux. Virtual memory is a strategy proposed to meet the shortage of physical memory. It uses disk space A piece of logical memory that is virtualized, and the disk space used as virtual memory is called swap space (Swap Space).
As an extension of physical memory, Linux will use the virtual memory of the swap partition when the physical memory is insufficient. More specifically, the kernel will write the temporarily unused memory block information to the swap space, so that the physical memory is released. This memory can then be used for other purposes, and when the original content is needed, the information will be re-read from the swap space into physical memory.
The memory management of Linux adopts the paging access mechanism. In order to ensure that the physical memory can be fully utilized, the kernel will automatically exchange the infrequently used data blocks in the physical memory into the virtual memory at an appropriate time, and the frequently used data blocks Information is persisted to physical memory.
To understand the linux memory operation mechanism in depth, you need to know the following aspects:
First of all, the Linux system will perform page swap operations from time to time to maintain as much free physical memory as possible. Even if there is nothing that requires memory, Linux will also swap out temporarily unused memory pages. This avoids the time required to wait for the exchange.
Secondly, linux performs page swapping conditionally. Not all pages are swapped to virtual memory when not in use. The linux kernel only swaps some infrequently used page files to virtual memory according to the "recently most frequently used" algorithm. Sometimes we You will see such a phenomenon: Linux still has a lot of physical memory, but the swap space is also used a lot. In fact, this is not surprising. For example, when a process that occupies a large amount of memory is running, it needs to consume a lot of memory resources. When the process ends and a lot of memory is released, the page file that was swapped out just now will not be automatically swapped into the physical memory. Unless it is necessary, the physical memory of the system will be much free at this moment, and the swap space is also being used at the same time. The phenomenon just mentioned appeared. On this point, don't worry about anything, just know what's going on.
Finally, the pages of the swap space will be swapped to physical memory first when they are used. If there is not enough physical memory to hold these pages at this time, they will be swapped out immediately. In this way, there may not be enough space in virtual memory to store These exchanged pages will eventually lead to problems such as false crashes and service exceptions in linux. Although linux can recover by itself within a period of time, the recovered system is basically unusable.
Therefore, it is very important to plan and design the use of Linux memory reasonably.
Two memory monitoring
As a linux system administrator, it is very important to monitor the usage status of memory. Monitoring helps to understand the usage status of memory, such as whether the memory usage is normal, whether the memory is in short supply, etc. The most commonly used commands for monitoring memory are free , top, etc., the following is the output of a system free:
[haixigov@WEBServer ~]$ free
total used free shared buffers cached
Mem: 16402432 16360492 41940 0 465404 12714880
-/+ buffers/cache: 3180208 13222224
Swap: 8193108 264 8192844
Let's explain the meaning of each option in the output results:
first is the first line:
total : the total size of physical memory.
used : How much physical memory has been used.
free : Free physical memory value.
shared : memory value shared by multiple processes.
buffers/cached : the size of the disk cache.
The second line Mem : represents the physical memory usage.
The third line ( -/+ buffers/cached ): represents the disk cache usage status.
The fourth line: Swap indicates the memory usage status of the swap space.
The memory status output by the free command can be viewed from two angles: one is from the perspective of the kernel, and the other is from the perspective of the application layer.
1. View the state of memory from the kernel's point of view
That is, the kernel can be allocated directly without additional operations. It is the value of the Mem item in the second line of the above free command output. It can be seen that the physical memory of this system is 16G, and the free memory is only 41940K, which is more than 40M. One point, let's do a calculation like this:
16402432－16360492＝41940
is actually the total physical memory minus the used physical memory to get the free physical memory size. Note that the available memory value 41940 here does not include buffers and The memory size of the cached state.
If you think that the free memory of this system is too small, then you are wrong. In fact, the kernel completely controls the usage of memory. Linux will save the status of buffers and cached when memory is needed, or when the system is running gradually. The memory becomes free state memory for system use.
2. View the usage status of system memory from the perspective of application layer
That is, the memory size that can be used by applications running on Linux, that is, the output of the third line "(-/+ buffers/cached)" of the free command. It can be seen that the memory used by this system is only 3180208K, and the free memory Reach 13222224K, continue to do such a calculation:
41940 + (465404 + 12714880) = 13222224
Through this equation, we can know that the physical memory value available to the application is the free value of the Mem item plus the sum of the buffers and cached values, that is to say, the free value It includes the size of buffers and cached items.
For applications, the memory occupied by buffers/cached is available, because buffers/cached is to improve the performance of file reading. When the application needs to use memory, buffers/cached cached will be recycled quickly for application use.
3. The similarities and differences between buffers and cached
In the Linux operating system, when the application needs to read the data in the file, the operating system first allocates some memory, reads the data from the disk into the memory, and then distributes the data to the application; When writing data, the operating system first allocates memory to receive user data, and then writes the data from memory to disk. However, if a large amount of data needs to be read from the disk to the memory or written to the disk from the memory, the read and write performance of the system will become very low, because whether it is reading data from the disk or writing data to the disk, it is a very slow process. A process that consumes time and resources. In this case, linux introduces buffers and cached mechanisms.
Both buffers and cached are memory operations, which are used to save the files and file attribute information that the system has opened, so that when the operating system needs to read some files, it will first search in the buffers and cached memory areas, and if found, read them out directly It is sent to the application program. If the required data is not found, it will be read from the disk. This is the caching mechanism of the operating system. Through caching, the performance of the operating system is greatly improved. But the contents of buffers and cached buffers are different.
Buffers are used to buffer block devices. It only records the metadata of the file system and tracking in-flight pages, while cached is used to buffer files. To put it more generally: buffers are mainly used to store what is in the directory, the attributes and permissions of the file, and so on. The cached is directly used to remember the files and programs we have opened.
In order to verify whether our conclusion is correct, you can open a very large file through vi to see the changes in the cache, and then vi this file again to feel the similarities and differences between the two opening speeds, whether the speed of the second opening is obvious Faster than the first time?
Then execute the following command:
find /* -name *.conf
See if the value of buffers changes, and then execute the find command repeatedly to see the difference in the display speed twice. The memory operation principle of the Linux operating system is largely designed according to the needs of the server. For example, the system's buffer mechanism will cache frequently used files and data in cached. Linux is always striving to cache more data And information, so that when the data is needed again, it can be directly fetched from the memory without a long disk operation. This design idea improves the overall performance of the system