Python entry tutorial 14-01 (Python memory leak of python syntax entry)

In the development environment, we usually seldom pay attention to the memory problem when writing python programs. Maybe friends who write c and c++ will consider this problem more, but once our python program has a memory leak problem, it will also be A very troublesome thing, then how should we avoid this kind of problem, if it happens, how to solve it, let's take a look!

Cause

Memory leak refers to the situation where the program fails to release the memory that is no longer used due to negligence or error. Memory leak does not mean the physical disappearance of the memory, but after the application allocates a certain segment of memory, due to a design error, it loses control of this segment of memory, which causes a waste of memory. Severe consequences such as slowing down the program and even crashing the system. Circular references between objects with del() function are the main cause of memory leaks

Program

When not using an object, use: delobject to delete the reference count of an object to effectively prevent memory leaks. Use the Python extension module gc to view detailed information about objects that cannot be recycled. You can get the reference count of the object through sys.getrefcount(obj), and determine whether the memory leak is based on whether the return value is 0.

However, due to the garbage collection mechanism of gc, it is necessary to traverse all python objects managed by the garbage collector (including garbage and non-garbage objects). This process is time-consuming and may cause program freezes, which will require higher memory and cpu requirements. The scenarios cause performance impact. So how can we elegantly avoid memory leaks?

Write safe code

For example, for the cycle_ref function that has a memory leak below, removing the circular reference before the end of the function can solve the memory leak problem.

def cycle_ref():

a1 = A()

a2 = A()

a1.child = a2

a2.child = a1

# 解除循环引用，避免内存泄露

a1.child  = None

a2.child  = None

But for the above method, we may forget those one or two lines of irrelevant code and cause catastrophic consequences. After all, the tiger also has a nap. What to do then? Don't worry, Python has already considered this for us: weak references.

Weak reference

The Python standard library provides the weakref module. Weak references are not counted in the reference count. The main purpose is to resolve circular references. Not all objects support weakref, such as list and dict. The following are the more commonly used methods of weakref:

"""

1. class weakref.ref(object[, callback]): create a weak reference object, object is the referenced object, callback is the callback function (when the referenced object is deleted, the callback function is called)

2. weakref.proxy(object[, callback]): Create a proxy object implemented with weak references, with the same parameters as above

3.weakref.getweakrefcount(object): Get the number of weak reference objects associated with the object object

4.weakref.getweakrefs(object): Get a list of weak reference objects associated with object

5.class weakref.WeakKeyDictionary([dict]): Create a dictionary whose key is a weak reference object

6.class weakref.WeakValueDictionary([dict]): Create a dictionary whose value is a weak reference object

7.class weakref.WeakSet([elements]): create a collection object whose members are weak reference objects

"""

Similarly, for the cycle_ref function where the memory leak occurs above, using weakref to modify it slightly can solve the memory leak problem more safely:

import weakref

class A(object):

def __init__(self):

    self.data = [x for x in range(100000)]

    self.child = None

def __del__(self):

    pass

def cycle_ref():

a1 = A()

a2 = A()

a1.child = weakref.proxy(a2)

a2.child = weakref.proxy(a1)

if name == 'main':

import time

while True:

    time.sleep(0.5)

    cycle_ref()