【2023】java memory overflow and memory leak code testing and detection

1. Basic concepts

• Memory overflow :
  • Simply put, memory overflow means that the memory requested during the running of the program is greater than the memory that the system can provide, resulting in the inability to apply for enough memory , so a memory overflow occurs.
  • An out of memory exception will occur ;
• Memory leak :
  • Memory leak means that memory is allocated to temporary variables during the running of the program . After use, it is not reclaimed by GC. It always occupies the memory and cannot be used or allocated to other programs. This is called a memory leak (that is, it is equivalent to occupying memory but not allocating it to other programs). cannot be managed to the point of wasting memory)
  • Memory leaks will not have much impact in the short term or even minor ones, but when memory leaks accumulate, they will be serious and will continue to occupy the available memory, resulting in memory overflow .

1. Memory leak

The root cause of memory leaks is that long-lived objects hold references to short-lived objects. Although the short-lived objects are no longer needed, they cannot be recycled because the long-lived objects hold references to them.
Classified by the way it occurs, memory leaks can be divided into 4 categories:

1. Frequent memory leaks. Code with memory leaks will be executed multiple times, causing a memory leak every time it is executed.
2. Sporadic memory leaks. Code that leaks memory only occurs under certain circumstances or operating procedures. Frequent and sporadic are relative. For a specific environment, sporadic may become regular. So the test environment and test method are crucial to detect memory leaks.
3. One-time memory leak. The code that causes a memory leak will only be executed once, or due to algorithmic flaws, there will always be only one and only one block of memory leaked. For example, if memory is allocated in the constructor of a class, but the memory is not released in the destructor, the memory leak will only occur once.
4. Implicit memory leak. The program continuously allocates memory while it is running, but does not release the memory until the end. Strictly speaking, there is no memory leak here, because the program eventually releases all the requested memory. But for a server program that needs to run for days, weeks or even months, failure to release memory in time may also lead to the eventual exhaustion of all the system's memory. Therefore, we call this type of memory leak an implicit memory leak.

From the perspective of users using the program, memory leaks themselves will not cause any harm. As ordinary users, > > they cannot feel the existence of memory leaks at all. What's really harmful is the accumulation of memory leaks, which will eventually consume all the system's memory. From this perspective, one-time memory leaks are not harmful because they do not accumulate, while implicit memory leaks are very harmful because they are more difficult to detect than recurrent and sporadic memory leaks. .

2.1. Memory leaks caused by static collection classes:

Memory leaks are most likely to occur when using HashMap, Vector, etc. The life cycle of these static variables is consistent with the application, and all the objects they refer to cannot be released, causing memory leaks because they will always be used by Vector, etc. Quote.

Vector<Object> v=new Vector<Object>(100);
for (int i = 1; i<100; i++)
{
    
    
Object o = new Object();
v.add(o);
o = null;
}

In this example, the Object object is allocated cyclically and the requested object is put into a Vector. If only the reference itself is released (o=null), the Vector still refers to the object, so this object is not recyclable by the GC. of. Therefore, if an object must be deleted from the Vector after it is added to the Vector, the simplest method is to set the Vector object to null.

2.2. Modify the parameter value of the object in the HashSet, and the parameter is the field for calculating the hash value

When an object is stored in the HashSet collection, and the fields in the object that participate in calculating the hash value are modified, the hash value of the object is different from the one originally stored in the collection. In this case, use contains The method cannot find the object when retrieving it from the collection. This will result in the failure to delete the current object from the HashSet, causing a memory leak. For example:

public static void main(String[] args){
    
    
        Set<Person> set = new HashSet<>();
        Person p1 = new Person("张三","1",25);
        Person p2 = new Person("李四","2",26);
        Person p3 = new Person("王五","3",27);

        set.add(p1);
        set.add(p2);
        set.add(p3);
        System.out.println("总共有:"+set.size()+" 个元素!"); //结果：总共有:3 个元素!

        System.out.println("修改前的哈希值："+p3.hashCode());
        p3.setAge(2); //修改p3的年龄,此时p3元素对应的hashcode值发生改变
        System.out.println("修改后的哈希值："+p3.hashCode());
        
        set.remove(p3); //此时remove不掉，造成内存泄漏
        set.add(p3); //重新添加，可以添加成功
        
        System.out.println("总共有:"+set.size()+" 个元素!"); //结果：总共有:4 个元素!
        for (Person person : set){
    
    
            System.out.println(person);
        }
}

Log result printing: the hashcode has been changed before and after modification.

2.3. Various connections

For example, database connections (dataSourse.getConnection()), network connections (socket) and io connections will not be automatically recycled by GC unless their close() method is explicitly called to close the connection. The Resultset and Statement objects do not need to be explicitly recycled, but the Connection must be explicitly recycled, because the Connection cannot be automatically recycled at any time, and once the Connection is recycled, the Resultset and Statement objects will immediately be NULL. But if you use a connection pool, the situation is different. In addition to explicitly closing the connection, you must also explicitly close the Resultset Statement objects (if you close one of them, the other one will also be closed). Otherwise, a large number of Statement objects will not be able to be closed. released, causing a memory leak. In this case, the connection is usually made in try and the connection is released in finally.

2.4. Singleton mode

If the singleton object holds a reference to an external object, the external object will not be recycled normally by the jvm, resulting in a memory leak.

  不正确使用单例模式是引起内存泄露的一个常见问题，单例对象在被初始化后将在JVM的整个生命周期中存在（以静态变量的方式），如果单例对象持有外部对象的引用，那么这个外部对象将不能被jvm正常回收，导致内存泄露，考虑下面的例子：

class A{
    
    
    public A(){
    
    
        B.getInstance().setA(this);
    }
    ....
}
//B类采用单例模式
class B{
    
    
    private A a;
    private static B instance=new B();
    public B(){
    
    }
    
    public static B getInstance(){
    
    
        return instance;
    }
    
    public void setA(A a){
    
    
        this.a=a;
    }
    //getter...
}

2. Common situations of memory overflow

Memory overflow has the following common situations:

1.1, java.lang.OutOfMemoryError: PermGen space (persistent with overflow)

We know that jvm implements the method area in the java virtual machine specification through persistence, and the runtime constant pool is stored in the method area. Therefore, this kind of overflow may be due to the runtime constant pool overflow, or due to the use of There are a large number of jars or classes, so the class objects saved in the method area are not recycled in time or the memory occupied by the class information exceeds the configured size.

1.2. java.lang.OutOfMemoryError: Java heap space (heap overflow)

The reason for this kind of overflow is generally that too many objects are created, and the number of objects reaches the capacity limit of the maximum heap before garbage collection.
The way to solve anomalies in this area is generally to use a memory image analysis tool to analyze the heap dump snapshot produced by Dump to see whether it is a memory overflow or a memory leak. If it is a memory leak, you can further use tools to view the reference chain from the leaked object to GC Roots, locate the location of the leaked code, and modify the program or algorithm; if there is no leak, which means that the objects in the memory must still survive, then You should check the heap parameters -Xmx (maximum heap size) and -Xms (initial heap size) of the virtual machine, and compare them with the physical memory of the machine to see if they can be increased.

• Code implementation:
  First set the heap parameters of the virtual machine: -Xmx (maximum heap size) and -Xms (initial heap size)
  
• test code

    public static void main(String[] args) {
    
    
        List list = new ArrayList();
        for (int i = 0; i < 1024; i++) {
    
    
            System.out.println(i);
            list.add(new byte[1024 * 1024]);
        }
    }

1.3. Virtual machine stack and local method stack overflow

If the stack depth requested by the thread is greater than the maximum depth allowed by the virtual machine, a StackOverflowError will be thrown.

If the virtual machine cannot apply for enough memory space when expanding the stack, an OutOfMemoryError is thrown.

How to detect memory overflow situations

JProfiles

If you are using idea, you can use JProfiler with the plug-in or app to detect and monitor

• You can view the memory usage of the implementation and the size of the memory occupied by the object.
  

Chinese version download link:
Link: https://pan.baidu.com/s/1Qjhx7A7x6vK3VBy5ns2Udg?pwd=8omj
Extraction code: 8omj

MAT

• If you are using eclipse, you can use the corresponding MAT tool for detection and monitoring.