Memory overflow and memory leak
What is a memory leak?
内存泄漏（存储渗漏）是用动态存储分配函数动态开辟的空间，在使用完毕后未释放，结果导致一直占据该内存单元，直到程序结束。发生内存泄漏的代码只有在某些特定环境或操作过程下才会发生。
1. The analogy of the memory leak image is "The storage space that the operating system can provide to all processes is being squeezed out by a certain process." The final result is that the longer the program runs, the more storage space it takes up, and all the storage is eventually used up. Space, the entire system collapsed. So the "memory leak" is from the perspective of the operating system. The storage space here does not refer to physical memory, but refers to the size of virtual memory. The size of this virtual memory depends on the size of the disk swap area. A piece of memory requested by the program without any pointer pointing to it, then this piece of memory It leaked.
2. In general, the memory leak we often say refers to the leak of heap memory . Heap memory refers to the memory that is allocated by the program from the heap and is of arbitrary size (the size of the memory block can be determined during the running of the program) and must be explicitly released after use.
3. Applications generally use malloc, realloc, new and other functions to allocate a piece of memory from the heap. After use, the program must be responsible for calling free or delete to release the memory block, otherwise, this memory cannot be used again. Let's say that this memory leaked.
4. Memory leaks have a lot to do with the reference count of objects. In addition, there is no automatic garbage collection mechanism in c/c++. If there is no manual memory release, problems will occur. If you want to avoid this problem, you still have to start with the code. Good coding habits and norms are the only way to avoid mistakes.
Memory leaks can be divided into 4 categories in the way they occur:
Frequent
The code that has a memory leak will be executed multiple times, and each time it is executed, it will cause a memory leak.
Sporadic
The code that has a memory leak can only happen under certain circumstances or operations. Frequent and sporadic are relative. For a specific environment, the occasional may become frequent. Therefore, the test environment and test methods are critical to detecting memory leaks.
One time
The code that has a memory leak will only be executed once, or due to algorithmic flaws, there will always be one and only one memory leak. For example, memory is allocated in the constructor of the class, but the memory is not released in the destructor, so the memory leak will only occur once.
Implicit
The program keeps allocating memory while it is running, but does not release the memory until the end. Strictly speaking, there is no memory leak, because the final program releases all the requested memory. But for a server program, it needs to run for days, weeks, or even months. Failure to release the memory in time may eventually exhaust all the system memory. Therefore, we call this type of memory leak an implicit memory leak.
What is a memory overflow?
内存溢出(Out Of Memory，简称 OOM)是指应用系统中存在无法回收的内存或使用的内存过多，最终使得程序运行要用到的内存大于能提供的最大内存。此时程序就运行不了，系统会提示内存溢出。
Memory overflow errors can cause the task of processing data to fail, and even cause serious consequences such as platform crashes.
Most of the processing methods for memory overflow are to re-execute the task. However, for memory overflow errors caused by system configuration, data flow, user code, etc., even if the user re-executes the task, it is still unavoidable.
The relationship between memory overflow and memory leak
Memory leak is one of the causes of memory overflow, but memory leak does not necessarily cause memory overflow . Simply put, memory overflow means that it takes up too much memory and exceeds the range that the system can withstand; while memory leaks are due to the delayed collection of objects allocated for program operation or even failure to be collected. Over time, the heap allocated by the system There are many useless references in the space.
From the point of view of the user's use of the program, the memory leak itself does not cause any harm, as a general user, there is no memory leak at all. What is really harmful is the accumulation of memory leaks, which will eventually exhaust all the system memory. From this point of view, a one-time memory leak is not harmful because it will not accumulate, while implicit memory leaks are very harmful because it is more difficult to detect than frequent and sporadic memory leaks. .
Causes of memory overflow
1. The amount of data loaded in the memory is too large, such as fetching too much data from the database at one time.
2. Memory leaks.
3. There is an infinite loop in the code or an object entity with too many repetitions.
4. BUG in the third-party software used.
5. The memory value of the startup parameter is set too small.
Memory overflow solution
plan 1:
Check the error log to see if there are other exceptions or errors before the "OutOfMemory" error. By carefully reviewing the log and analyzing what operations were done before the memory overflow, the problematic module can be roughly located.
Scenario 2:
Walk through and analyze the code to find out where memory overflow may occur. Focus on the following points:
- Check the code for infinite loops or recursive calls. A large number of recursive calls will create a copy of the function, which will consume a lot of time and memory.
- Check whether there is a big cycle repeatedly generating new object entities.
- Check whether there is a query to obtain all the data in the database query. Generally speaking, if one hundred thousand records are fetched into the memory at a time, it may cause memory overflow. This problem is relatively hidden. In the early stage of software operation, there is less data in the database, which is not easy to cause problems. As the running time increases, the amount of data in the database also increases, and a query may cause memory overflow. Therefore, for database queries, try to use paged queries.
- Check whether the collection objects such as List and MAP are not cleared after use. Collection objects such as List and MAP will always have references to objects, so that these objects cannot be recycled by GC.
Option 3:
Use the memory viewing tool to dynamically view the memory usage. For slow memory leaks, the software may be normal in the initial stage of operation, but because the memory has been applied for and not released during the operation, the system resources are exhausted and the software crashes and other problems. This kind of slow memory leak cannot be solved by the above two solutions, which requires the use of memory viewing tools to monitor the memory usage in real time.