Solution to report Memoryerror during Python program running

In solving a high-dimensional time-dependent nonlinear PDE, the spatial calculation domain is M=256 in each spatial direction, and a fast algorithm is implemented to solve the problem of storage and calculation speed; while the time direction is T=2000, the time step Long 0.01.

It has been run in advance in Spyder. If it iterates 1468 times for the first time, it will report Memoryerror

Note that all defined arrays in the program can be applied for under limited storage

Then close Spyder, re-run, for example, it can iterate 12147 times, and report Memoryerror

First, you can check whether the computer memory is insufficient through the task manager
insert image description here

Then, according to some existing solutions, we try to deal with the following:
1. Expand the virtual memory.
I found in the process of running the code that when a memory error error occurs, it is impossible for personal storage to have this error, so I checked it. , It is found that the memory is limited, consider turning off some software that may limit the memory, expand the virtual memory, and so on.

The method of expanding virtual memory (my system is win8, but it should be similar):
1. Open the control panel;
2. Find the item of system;
3. Find the item of advanced system settings;
4. Click the setting button of the performance module ;
5. Select the advanced panel, click Change in the virtual memory module;
6. Remember not to select "Automatically manage the paging file size of all drives", and then select a drive, that is, a disk, select the custom size, and manually enter the initial size and The maximum value, of course, it is best not to be too large, and you can check the usage of the disk after the change, so as not to lose too much space.
7. After all settings are completed, remember to click "Settings" and then confirm, otherwise it will be invalid, and finally restart the computer.

If memory errors still occur after setting this step, then

2. Update the Pandas and Numpy libraries to 64-bit.
If your Python uses 32-bit, then your pandas and Numpy can only be 32-bit. Then when your memory usage exceeds 2G, the memory will be automatically terminated . I found this error because when I noticed the memory overflow error, my memory clearly showed that only 40+% was used, and then the error message was in the core of pandas, so I checked and found that there is still such a large pit.

The solution is: first check how many bits your python is, enter python in the shell, and check the number of bits. If it is 32 bits, then reinstall Python and install a 64-bit one, but at the same time your library also needs to be reinstalled. Installed. But after I executed this step, the problem was perfectly solved!

3. If there is still a memory overflow error
, use the above methods one by one in order, and stop when the error disappears. Of course, if your memory display usage reaches 99%+, then the memory is really not enough, not other problems, if it is not a particularly large amount of data, it is a matter of habit when writing code, although Python has garbage collection Mechanism, but sometimes it may be too late to recycle, especially in the process of loop iterations, it is often time to clean up the garbage after the loop is over, so remember to delete the unnecessary variables in time, or use the garbage collection library of gc, so that the memory will naturally be It’s always refreshing~
4. Change the desktop to perform calculations

insert image description here

Guess you like

Origin blog.csdn.net/y15520833229/article/details/130589639