1. mtrace analyzes memory leaks
mtrace (memory trace) is a memory problem detection tool that comes with GNU Glibc. It can be used to help locate memory leak problems. Its implementation source code is in the malloc directory of the glibc source code. Its basic design principle is to design a function void mtrace (). The function traces the calls of malloc/free and other functions in the libc library, thereby detecting whether there is a memory leak. Condition. mtrace is a C function, declared and defined in <mcheck.h>. The function prototype is:
void mtrace(void);
mtrace principle
mtrace() The function will install "hook" functions for those functions related to dynamic memory allocation (such as malloc(), realloc(), memalign() and free()). These hook functions will record all relevant memory allocation and The tracking information released, and muntrace() will unload the corresponding hook function.
Based on the debugging trace information generated by these hook functions, we can analyze whether there are problems such as "memory leaks".
Set log generation path
The mtrace mechanism requires us to actually run the program before it can generate trace logs, but one more thing to do before actually running the program is to tell mtrace (the hook function mentioned above) the path to generate the log file.
There are two ways to set the log generation path, one is to set the environment variable: export MALLOC_TRACE=./test.log // 当前目录下 the other is to set it at the code level: setenv("MALLOC_TRACE", "output_file_name", 1);``output_file_nameit is the name of the file that stores the detection results.
Test example
#include <mcheck.h>
#include <stdlib.h>
#include <stdio.h>
int main(int argc, char **argv)
{
mtrace(); // 开始跟踪
char *p = (char *)malloc(100);
free(p);
p = NULL;
p = (char *)malloc(100);
muntrace(); // 结束跟踪，并生成日志信息
return 0;
}
From the above code, we hope to be able to check whether there is a memory leak from the beginning to the end of the program. The example is simple. You can see at a glance that there is a memory leak, so we need to verify whether mtrace can check for memory leaks, and check How to analyze and position the results. gcc -g test.c -o testGenerate executable file.
log
After the program is finished running, the test.log file will be generated in the current directory. When you open it, you can see the following content:
= Start
@ ./test:[0x400624] + 0x21ed450 0x64
@ ./test:[0x400634] - 0x21ed450
@ ./test:[0x400646] + 0x21ed450 0x64
= End
From this file, we can see that the three lines in the middle correspond to the malloc -> free -> malloc operations in the source code. Interpretation : ./test refers to the name of the program we execute, [0x400624] is the address information in the machine code of the first call to the malloc function, + means applying for memory (- means releasing), 0x21ed450 is the address information applied for by the malloc function , 0x64 represents the requested memory size. From this analysis, the first application has been released, but the second application has not been released, and there is a memory leak problem.
Leak analysis
Use the addr2line tool to locate the source code location
By using the "addr2line" command tool, you can get the line number of the source file (you can use this to locate the specific source code location based on the machine code address)
# addr2line -e test 0x400624
/home/test.c:9
Use the mtrace tool to analyze log information
mtrace + executable file path + log file path mtrace test ./test.logare executed, and the following information is output:
Memory not freed:
-----------------
Address Size Caller
0x00000000021ed450 0x64 at /home/test.c:14
2. Valgrind analyzes memory leaks
Valgrind tool introduction
Valgrind is a collection of open source (GPL V2) simulation and debugging tools under Linux. Valgrind consists of a core and other debugging tools based on the core. The kernel is similar to a framework, which simulates a CPU environment and provides services to other tools; other tools are similar to plug-ins, using the services provided by the kernel to complete various specific memory debugging tasks. The architecture of Valgrind is shown in the figure below
1、Memcheck
The most commonly used tool is used to detect memory problems in programs. All reads and writes to memory will be detected, and all calls to malloc() / free() / new / delete will be captured.
Therefore, it can detect the following problems: use of uninitialized memory; reading/writing freed memory blocks; reading/writing memory blocks beyond malloc allocation; reading/writing inappropriate memory blocks in the stack; memory leaks, pointing to a block Memory pointers are lost forever; incorrect malloc/free or new/delete matching; dst and src pointers in memcpy() related functions overlap.
2、Callgrind
An analysis tool similar to gprof, but it is more detailed in observing the running of the program and can provide us with more information. Unlike gprof, it does not require special options when compiling the source code, but adding debugging options is recommended.
Callgrind collects some data when the program is running, builds a function call graph, and can optionally perform cache simulation. At the end of the run, it writes the analysis data to a file. callgrind_annotate can convert the contents of this file into a readable form.
3、Cachegrind
Cache analyzer, which simulates the first-level cache I1, Dl and second-level cache in the CPU, can accurately point out cache misses and hits in the program. If needed, it can also provide us with the number of cache misses, the number of memory references, and the number of instructions generated by each line of code, each function, each module, and the entire program. This is a great help for optimizing programs.
4、Helgrind
It is mainly used to check competition problems that occur in multi-threaded programs. Helgrind looks for areas in memory that are accessed by multiple threads and are not consistently locked. These areas are often where synchronization between threads is lost, and can lead to hard-to-find errors.
Helgrind implemented a race detection algorithm called "Eraser" and made further improvements to reduce the number of reported errors. However, Helgrind is still in the experimental stage.
5、Massif
Stack analyzer, which measures how much memory a program uses in the stack, tells us the size of the heap blocks, heap management blocks and stack.
Massif can help us reduce memory usage. In modern systems with virtual memory, it can also speed up the running of our programs and reduce the chance of the program staying in the swap area.
Additionally, lackey and nulgrind will be provided. Lackey is a small tool that is rarely used; Nulgrind just shows developers how to create a tool.
Information Direct: Linux kernel source code technology learning route + video tutorial kernel source code
Learning Express: Linux Kernel Source Code Memory Tuning File System Process Management Device Driver/Network Protocol Stack
Memcheck principle
The focus of this article is on detecting memory leaks, so I won’t explain too much about other tools of valgrind. I mainly explain the work of Memcheck. The principle of Memcheck detecting memory problems is shown in the figure below:
The key to Memcheck's ability to detect memory problems is that it creates two global tables.
- The Valid-Value table has 8 bits corresponding to each byte in the entire address space of the process; there is also a corresponding bit vector for each register of the CPU. These bits are responsible for recording whether the byte or register value has a valid, initialized value.
- The Valid-Address table has a corresponding bit for each byte in the entire address space of the process, which is responsible for recording whether the address can be read or written.
- Detection principle: When you want to read or write a byte in the memory, first check the A bit in the Valid-Address table corresponding to this byte. If the A bit shows that the location is an invalid location, memcheck reports a read and write error. The core is similar to a virtual CPU environment, so when a certain byte in the memory is loaded into the real CPU, the V bit in the Valid-Value table corresponding to the byte is also loaded into the virtual CPU Environment. Once the value in the register is used to generate a memory address, or the value can affect program output, memcheck will check the corresponding V bits. If the value has not been initialized, an uninitialized memory error will be reported.
Memory leak type
valgrind divides memory leaks into 4 categories:
- Definitely lost: The memory has not been released, but there is no pointer pointing to the memory, and the memory cannot be accessed. It is established that leaked running memory is strongly required to be patched.
- Indirectly leaked (indirectly lost): The leaked memory pointer is stored in the memory that has been leaked. As the memory that has been leaked cannot be accessed, the memory that caused the indirect leak cannot be accessed. For example:
struct list {
struct list *next;
};
int main(int argc, char **argv)
{
struct list *root;
root = (struct list *)malloc(sizeof(struct list));
root->next = (struct list *)malloc(sizeof(struct list));
printf("root %p roop->next %p\n", root, root->next);
root = NULL;
return 0;
}
What is missing here is the root pointer (which is the established leak type), causing the next pointer stored in the root to become an indirect leak. Indirectly leaked memory will definitely need to be patched, but it will usually be patched along with the patching of the established leak.
- Possibly lost: the needle does not point to the memory header address, but to the location inside the memory. Valgrind often suspects that there may be a leak because the hands are already biased and are not biased toward the memory head, but are biased toward the internal parts of the memory. In some cases, this is not a leak, because this program is designed that way, for example, in order to achieve memory alignment, additional application processing memory is returned to the aligned memory address.
- Still reachable: The pointer is always present and tilted toward the top of the memory, and the memory has not been released until the program exits.
Valgrind parameter settings
- --leak-check=<no|summary|yes|full> If set to yes or full, after the called program ends, valgrind will describe each memory leak in detail. The default is summary, which only reports several memory leaks.
- --log-fd= [default: 2, stderr] valgrind prints logs and dumps them to the specified file or file descriptor. Without this parameter, valgrind's logs will be output together with the user program's logs, which will appear very messy.
- --trace-children=<yes | no> [default: no] Whether to trace child processes. If it is a multi-process program, it is recommended to use this function. However, it will not have much impact if a single process is enabled.
- --keep-debuginfo=<yes | no> [default: no] If the program uses a dynamically loaded library (dlopen), the debug information will be cleared when the dynamic library is unloaded (dlclose). After enabling this option, the call stack information will be retained even if the dynamic library is unloaded.
- --keep-stacktraces=<alloc | free | alloc-and-free | alloc-then-free | none> [default: alloc-and-free] Memory leaks are nothing more than mismatching of application and release, and the function call stack is only Record when applying, or record when applying for release. If we only focus on memory leaks, there is actually no need to record both when applying for release, because this will occupy a lot of extra memory and more CPU consumption, making the already slow execution The program adds insult to injury.
- --freelist-vol= When a client program uses free or delete to release a memory block, the memory block will not be immediately available for reallocation. It will only be placed in a free blocks queue (freelist) and marked as unavailable. Access, which is helpful for detecting errors when the client program accesses the released block after a very important period of time. This option specifies the byte block size occupied by the queue. The default is 20MB. Increasing this option will increase the memory overhead of memcheck, but the ability to detect such errors will also be improved.
- --freelist-big-blocks= When taking available memory blocks from the freelist queue for reallocation, memcheck will take out a block according to priority from those memory blocks larger than number. This option prevents frequent calls to small memory blocks in the freelist. This option increases the probability of detecting wild pointer errors for small memory blocks. If this option is set to 0, all blocks will be reallocated on a first-in, first-out basis. The default is 1M. Reference: Introduction to valgrind (memory checking tool)
Recommended compilation parameters
In order to print out the destack information in detail when a problem occurs, it is actually best to add the -g option when compiling the program. If there is a dynamically loaded library, it must be added --keep-debuginfo=yes . Otherwise, if it is found that the dynamically loaded library is leaked, the symbol table cannot be found because the dynamic library has been uninstalled. Code compiler optimization, it is not recommended to use -O2 and above. -O0 is likely to slow down the operation, so it is recommended to use -O1.
Detection example description
Apply for memory without releasing it
#include <stdlib.h>
#include <stdio.h>
void func()
{
//只申请内存而不释放
void *p=malloc(sizeof(int));
}
int main()
{
func();
return 0;
}
Use the valgrind command to execute the program and output the log to a file
valgrind --log-file=valReport --leak-check=full --show-reachable=yes --leak-resolution=low ./a.out
Parameter Description:
- –log-file=valReport specifies to generate an analysis log file in the current execution directory, and the file name is valReport
- –leak-check=full displays details of each leak
- –show-reachable=yes Whether to detect leaks outside the control range, such as global pointers, static pointers, etc., and display all memory leak types
- –leak-resolution=low memory leak report merging level
- –track-origins=yes means turning on the “use of uninitialized memory” detection function and opening detailed results. If there is no such sentence, detection in this area will be performed by default, but detailed results will not be printed. After executing the output, the report is interpreted. 54017 refers to the process number. If the program is executed using multiple processes, the contents of multiple processes will be displayed.
==54017== Memcheck, a memory error detector
==54017== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==54017== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==54017== Command: ./a.out
==54017== Parent PID: 52130
The second paragraph is a summary of heap memory allocation. It mentions that the program applied for memory once, of which 0 were released, and 4 bytes were allocated () 1 allocs, 0 frees, 4 bytes allocated.
In the head summary, there is the total amount of heap memory used by the program, the number of memory allocation times and the number of memory release times. If the number of memory allocation times and memory release times are inconsistent, it means there is a memory leak.
==54017== HEAP SUMMARY:
==54017== in use at exit: 4 bytes in 1 blocks
==54017== total heap usage: 1 allocs, 0 frees, 4 bytes allocated
The third paragraph describes the specific information of the memory leak. There is a piece of memory that occupies 4 bytes ( 4 bytes in 1 blocks). It is allocated by calling malloc. You can see in the call stack that the func function finally called malloc, so this information is relatively accurate. This locates where our leaked memory is applied for.
==54017== 4 bytes in 1 blocks are definitely lost in loss record 1 of 1
==54017== at 0x4C29F73: malloc (vg_replace_malloc.c:309)
==54017== by 0x40057E: func() (in /home/oceanstar/CLionProjects/Share/src/a.out)
==54017== by 0x40058D: main (in /home/oceanstar/CLionProjects/Share/src/a.out)
The last paragraph is a summary, a memory leak of 4 bytes.
==54017== LEAK SUMMARY:
==54017== definitely lost: 4 bytes in 1 blocks // 确立泄露
==54017== indirectly lost: 0 bytes in 0 blocks // 间接性泄露
==54017== possibly lost: 0 bytes in 0 blocks // 很有可能泄露
==54017== still reachable: 0 bytes in 0 blocks // 仍可访达
==54017== suppressed: 0 bytes in 0 blocks
Reading and writing beyond the boundaries
#include <stdio.h>
#include <iostream>
int main()
{
int len = 5;
int *pt = (int*)malloc(len*sizeof(int)); //problem1: not freed
int *p = pt;
for (int i = 0; i < len; i++){
p++;
}
*p = 5; //problem2: heap block overrun
printf("%d\n", *p); //problem3: heap block overrun
// free(pt);
return 0;
}
problem1: The pointer pt applied for space, but was not released; problem2: pt applied for the space of 5 ints, and when p reached the position of p[5] after 5 cycles, the access was out of bounds (the write was out of bounds) *p = 5. (Invalid write of size 4 in the valgrind report below)
==58261== Invalid write of size 4
==58261== at 0x400707: main (main.cpp:12)
==58261== Address 0x5a23054 is 0 bytes after a block of size 20 alloc'd
==58261== at 0x4C29F73: malloc (vg_replace_malloc.c:309)
==58261== by 0x4006DC: main (main.cpp:7)
problem1: read out of bounds (Invalid read of size 4 in the valgrind report below)
==58261== Invalid read of size 4
==58261== at 0x400711: main (main.cpp:13)
==58261== Address 0x5a23054 is 0 bytes after a block of size 20 alloc'd
==58261== at 0x4C29F73: malloc (vg_replace_malloc.c:309)
==58261== by 0x4006DC: main (main.cpp:7)
Repeated release
#include <stdio.h>
#include <iostream>
int main()
{
int *x;
x = static_cast<int *>(malloc(8 * sizeof(int)));
x = static_cast<int *>(malloc(8 * sizeof(int)));
free(x);
free(x);
return 0;
}
The report is as follows,Invalid free() / delete / delete[] / realloc()
==59602== Invalid free() / delete / delete[] / realloc()
==59602== at 0x4C2B06D: free (vg_replace_malloc.c:540)
==59602== by 0x4006FE: main (main.cpp:10)
==59602== Address 0x5a230a0 is 0 bytes inside a block of size 32 free'd
==59602== at 0x4C2B06D: free (vg_replace_malloc.c:540)
==59602== by 0x4006F2: main (main.cpp:9)
==59602== Block was alloc'd at
==59602== at 0x4C29F73: malloc (vg_replace_malloc.c:309)
==59602== by 0x4006E2: main (main.cpp:8)
The application release interface does not match
The report of application and release interface mismatch is as follows. The pointer to apply for space using malloc is released using free; the space applied for using new is released using delete( Mismatched free() / delete / delete []):
==61950== Mismatched free() / delete / delete []
==61950== at 0x4C2BB8F: operator delete[](void*) (vg_replace_malloc.c:651)
==61950== by 0x4006E8: main (main.cpp:8)
==61950== Address 0x5a23040 is 0 bytes inside a block of size 5 alloc'd
==61950== at 0x4C29F73: malloc (vg_replace_malloc.c:309)
==61950== by 0x4006D1: main (main.cpp:7)
memory overwrite
int main()
{
char str[11];
for (int i = 0; i < 11; i++){
str[i] = i;
}
memcpy(str + 1, str, 5);
char x[5] = "abcd";
strncpy(x + 2, x, 3);
}
The problem lies in memcpy. Copying 5 chars starting from the str pointer position to the space pointed by str+1 will cause memory overwriting. The same goes for strncpy. The report is as follows Source and destination overlap:
==61609== Source and destination overlap in memcpy(0x1ffefffe31, 0x1ffefffe30, 5)
==61609== at 0x4C2E81D: memcpy@@GLIBC_2.14 (vg_replace_strmem.c:1035)
==61609== by 0x400721: main (main.cpp:11)
==61609==
==61609== Source and destination overlap in strncpy(0x1ffefffe25, 0x1ffefffe23, 3)
==61609== at 0x4C2D453: strncpy (vg_replace_strmem.c:552)
==61609== by 0x400748: main (main.cpp:14)
3. Summary
There are two memory detection methods:
1. Maintain a memory operation linked list. When there is a memory application operation, it is added to this linked list. When there is a release operation, it is removed from the linked list from the application operation. If there is still content in the linked list after the program ends, it means that there is a memory leak; if the memory operation to be released does not find the corresponding operation in the linked list, it means that it has been released multiple times. Use this method with built-in debugging tools, Visual Leak Detecter, mtrace, memwatch, debug_new. 2. Simulate the address space of the process. Following the operating system's handling of process memory operations, an address space mapping is maintained in user mode. This method requires a deep understanding of the processing of process address spaces. Because the process address space distribution of Windows is not open source, it is difficult to simulate, so it is only supported on Linux. The one that takes this approach is valgrind.
Original author: Learn embedded together
