Analyzing the VIRT RES SHR value displayed by the top command

The exact meaning of VIRT RES SHR

 

Analyzing the VIRT RES SHR value displayed by the top command - yalung - YALUNG

 

What are the exact meanings of the three memory indicators, VRIT , RES , and SHR ? Who can tell us? MAN page? Linux expert? SUSE engineer? Linus ? Who can say the most correct answer? no one! Because only the source code is the most correct answer.

Then let's take a look at the source code. This is the biggest benefit of open source software.

First of all, the source of these three data must be the kernel, and the related data structures of the process must be maintained by the kernel. Then top , as a user space program, needs to obtain the data in the kernel space through the system interface ( API ). The proc file system is a way of exchanging data between the Linux kernel space and the user space, and it is a very important way, which is different from the function call-based form of Windows .

When you call the system function read to read a normal file, the kernel executes the code corresponding to the file system to transfer the contents of the file from the disk to you.

When you call the system function read to read a  proc file, the kernel executes the code of the corresponding proc file system to transfer the relevant content to you from the kernel's data structure. proc files have nothing to do with disk. Just the system interface.

And the relevant information of a process, Linux tells us all through the files in /proc/<pid>/ .

As follows, you can use common file reading and writing tools, such as cat , to obtain various information about the process. This is much more flexible and richer than function calls.

 

Analyzing the VIRT RES SHR value displayed by the top command - yalung - YALUNG

 

 

Back to our question, the process information displayed by the top command must also be obtained through proc , because there is no other way, there is no system function to do this, and it is impossible for top to go directly to the kernel to obtain data from the user layer.

With the above information, you can quickly find the key code from the source code of top :

 

Analyzing the VIRT RES SHR value displayed by the top command - yalung - YALUNG
 

 

statm file:

 

Analyzing the VIRT RES SHR value displayed by the top command - yalung - YALUNG

 

According to the order of sscanf , the first value is VIRT , the second value is RES , and the third value is SHR !

Wait, it seems that the values ​​do not match, the SHR displayed by top is 344k , and the statm gives 86 !

Let's look at another key line of code:

 

Analyzing the VIRT RES SHR value displayed by the top command - yalung - YALUNG
 

 

statm shows the number of pages, top shows the KB . Under X86 , a page is 4KB , 86 * 4 = 344 . now it's right!

 

So, we found the most critical entry, and then click on the map to find out how the kernel generates the content of the statm file. ~~

 

Analyzing the VIRT RES SHR value displayed by the top command - yalung - YALUNG

 

 

The proc_pid_statm function is responsible for generating the contents of the statm file. When you use the cat command to print the statm file, this function in the kernel will be executed.

proc_pid_statm obtains the mm_struct data structure of the process, and this data structure is the memory descriptor of the process, through which all information about the memory usage and mapping of the process can be obtained.

     Further inspection of the task_statm function shows that:

 

Analyzing the VIRT RES SHR value displayed by the top command - yalung - YALUNG

 

第一个值(VIRT)就是mm->total_vm,即进程虚存的总大小,这个比较清晰,只要进程申请了内存,无论是malloc还是堆栈还是全局,都会计入这个值;

第二个值(RES)是mm->file_rss+mm->anon_rss

第三个值(SHR)是mm->file_rss

 RES要和SHR结合者看,内核把物理内存分为了两部分,一部分是映射至文件的,一部分是没有映射至文件的即匿名内存,完全和共不共享没有关系!

file_rss为什么叫做shared呢?应该是一种指示性表述,表示这部分内存可能是共享的。但并不代表真正共享了。那么到底哪些计入file_rss?通过查阅相关代码,发现(可能有遗漏):

程序的代码段。

动态库的代码段。

通过mmap做的文件映射。

通过mmap做的匿名映射,但指明了MAP_SHARED属性。

通过shmget申请的共享内存。

 即进程通过以上方式占用的物理内存,计入file_rss,也就是topSHR字段。我们看到一般这些内存都是以共享方式存在。但如果某个动态库只一个进程在使用,它的代码段就没有被共享着。

反过来再来看anon_rss统计的内容,是否就一定是独占的?也不是,比如新fork之后的子进程,由于copy on write机制,在页面被修改之前,和父进程共享。这部分值并不体现在top命令的SHR字段内。

 综上所述top命令显示的SHR字段,并不是准确描述了进程与其他进程共享使用的内存数量,是存在误差的。 

那么如何获取进程准确的共享内存数量?

获取进程准确的共享内存数量

我们注意到在描述进程信息的proc/<pid>内,有一个smaps文件,里面展示了所有内存段的信息,其中有Shared_Clean Shared_Dirty Private_Clean Private_Dirty:几个字段

 

Analyzing the VIRT RES SHR value displayed by the top command - yalung - YALUNG
 

 

 

找到相关代码,可以看到,一个页面如果映射数>=2计入Shared_* ; 如果=1计入Private_*。(脏页计入*_Dirty,否则计入*_Clean

 

 

Analyzing the VIRT RES SHR value displayed by the top command - yalung - YALUNG

 

     统计smaps文件内所有段的Shared_*值的总和就是进程准确的共享内存数量!

     统计smaps文件内所有段的Private_*值的总和就是进程准确的独占内存数量!

总结

通过以上分析,我们可以得到如下结论:

top命令通过解析/proc/<pid>/statm统计VIRTRESSHR字段值。

VIRT是申请的虚拟内存总量。

RES是进程使用的物理内存总和。

SHRRES映射至文件的物理内存总和。包括:

程序的代码段。

动态库的代码段。

通过mmap做的文件映射。

通过mmap做的匿名映射,但指明了MAP_SHARED属性。

通过shmget申请的共享内存。

/proc/<pid>/smapsShared_*统计的是RES映射数量>=2的物理内存。

Private _* in /proc/<pid>/smaps counts the physical memory with the number of mappings = 1 in RES .

Guess you like

Origin http://10.200.1.11:23101/article/api/json?id=326694925&siteId=291194637