Python's memory management mechanism

At a shallower level, Python's memory management mechanism is mainly divided into the following three aspects:

1. garbage collection
2. reference counting
3. memory pool mechanism

1. Garbage collection:

　　Unlike languages ​​such as C++ and Java, Python can directly assign values ​​to variables without declaring the type of variables in advance. For the Python language, the type and memory of objects are determined at runtime. This is why we call Python a dynamic language (here we can simply boil down the dynamic type to the fact that the allocation of the memory address of the variable is to automatically determine the type of the variable and assign values ​​to the variable at runtime).

Second, the reference count:

　　Python manages memory in the same way as Windows kernel objects. Each object maintains a count of references to the object, as shown in the following figure (picture from Python core programming):

When we execute the following code:

x = 3.14 
y = x

　　First we will create an object 3.14, and then assign a reference to the float object to x, because x is the first reference, so the float object has a reference count of 1. The statement "y = x" creates a reference alias y that points to the same object. In fact, it does not create a new object for y, but points y to the floating-point object pointed to by x. At this time, the reference to the floating-point object is The count is 2.

　　We can demonstrate this with code:

>>> a = 10
>>> b = a
>>> id(a)
10418052
>>> id(b)
10418052

　　The id of variable a and variable b are the same (we can think of the id value as a pointer to a variable in C).

　　We cite a picture from another website to illustrate the problem: for C language, when we create a variable A, we will apply for a memory space for the variable, and put the variable value into the space, when the variable is assigned to Another variable B will apply for a new memory space for B, and put the variable value into the memory space of B, which is why the pointers of A and B are inconsistent. As shown in the figure:

　　The situation in Python is different. In fact, Python's processing is somewhat similar to Javascript. As shown in the figure, variables are more like labels attached to objects (similar to the definition of references). When a variable is bound to an object, the reference count of the variable is 1, (there are other situations that will also cause the variable reference count to increase), the system will automatically maintain these labels, and scan regularly, when a label The pair is reclaimed when its reference count reaches 0.

3. Memory pool mechanism:

Python's memory mechanism can be represented in pyramid rows:

• -1, -2 layers are mainly operated by the operating system;
• The 0th layer is the operation of memory allocation and release functions such as malloc and free in C;
• The first layer and the second layer are memory pools, which are implemented by the Python interface function PyMem_Malloc function. When the object is less than 256K, this layer directly allocates memory;
• The third layer is the top layer, which is our direct manipulation of Python objects;

　　In C, if you call malloc and free frequently, it will cause performance problems. Coupled with the frequent allocation and release of small blocks of memory, memory fragmentation will occur. The main tasks of Python here are:

• If the requested allocated memory is between 1 and 256 bytes, use your own memory management system, otherwise use malloc directly.
• Here, malloc will still be called to allocate memory, but each time a large block of memory with a size of 256k will be allocated.

　　The memory registered through the memory pool will still be recycled to the memory pool at the end, and will not be released by calling C's free. For next use. For simple Python objects, such as values, strings, tuples (tuple is not allowed to be changed ) uses the method of copying (deep copy?), that is to say, when another variable B is assigned to variable A, although the memory space of A and B is still the same, when the value of A changes, it will be re-assigned A allocates space, and the addresses of A and B are no longer the same:

And for things like dictionaries (dict), lists (List), etc., changing one will cause changes to the other, also known as shallow copy:

When the reference count is incremented:

1. Object is created: x=4
2. Others are created: y=x
3. is passed as an argument to the function: foo(x)
4. As an element of the container object: a=[1,x,'33']

The case where the reference count is reduced

1. A local reference goes out of its scope. For example, when the foo(x) function above ends, the object reference pointed to by x is decremented by 1.
2. Object aliases are explicitly destroyed: del x; or del y
3. An alias of an object is assigned to another object: x=789
4. Object removed from a window object: myList.remove(x)
5. The window object itself is destroyed: del myList, or the window object itself goes out of scope.

garbage collection

1. When there are parts of memory that are no longer used, the garbage collector cleans them up. It will check those objects whose reference count is 0 , and then clear their space in memory. Of course, in addition to the reference count of 0 will be cleared, there is also a situation will be cleared by the garbage collector: when two objects refer to each other, their other references are already 0.
2. The garbage collection mechanism also has a cyclic garbage collector that ensures the release of circular reference objects (a refers to b, and b refers to a, causing its reference count to never be 0).