As we all know, in the series of ArcGIS products, the python environment of Desktop is 32-bit, and the python environment of Server is 64-bit, and it uses python2 syntax. ArcGIS Pro uses the syntax of python3, and the python environment is also 64-bit. In contrast, Desktop's python environment will give people a feeling of inadequate use. In fact, it is true. When faced with a large amount of data, the 32-bit python environment has extremely limited use of memory resources. Yes, it will be easier to explode memory overflow errors.
For example, the officially recognized software vulnerability numbered NIM059548 is specifically manifested as:
arcpy.mapping Export and List functions leak memory and resource handles. This causes the host process for the script to crash or become unresponsive and limits the use of arcpy.mapping inside of GP services.
The website of the official document can be found at http://downloads2.esri.com/support/documentation/ao_/10.0_SP1_Announcement.pdf .
According to the official statement, the vulnerability has been fixed in the 10.1 version, but in my personal use, the vulnerability still exists in the 10.6 version.
But later, I accidentally discovered that the memory overflow problem will not occur when this function is executed in a 64-bit python environment. But the python environment that comes with Desktop is 32-bit, what should I do?
It's very simple. Find the official BackgroundGP installation package. After the installation is complete, you will have a 64-bit python runtime environment, which is completely independent of the 32-bit python environment and also has the arcpy library.
The python environment of ArcGIS Desktop is installed in the C:\Python27 directory by default. Open the directory again, you will find the directory structure is as follows:
these two directories correspond to the 32-bit and 64-bit python environments, and the folders contain python.exe.
Next time you run the script, just point the python running path to the 64-bit python environment.