Memray is an open source memory profiler developed by Bloomberg. It has been open source for more than a month and has already received over 8.4k stars, making it a veritable star project. Today we will recommend this Python memory analysis tool to everyone.
Memray can track memory allocations in Python code, native extension modules, and the Python interpreter itself. It can generate many different types of reports to help you analyze Python code memory usage.
-
Main features of the tool:
-
Track each function call and accurately trace the call stack
-
Can trace c/c++ library calls
-
Analysis is very fast
-
Collect memory data and output various icons
-
Using python threads
-
Work with native threads
-
Problems that can help solve:
-
Analyze memory allocations in applications to discover causes of high memory usage
-
Find the cause of the memory leak
-
Find code hotspots that cause heavy memory allocations
►►► memray installation
-
Environmental requirements: python3.7+ or above, Linux system (only supports Linux system)
-
Installation: pip3 install memray
►►► memray usage
memray usage help
python3 -m memray --help
| parameter |
effect |
| run |
Run the specified application and track memory usage |
| flamegraph |
In the html report, use flame graph to display memory usage |
| table |
In the html report file, display the memory analysis situation in a table |
| live |
Use real-time screen display to display various memory usage |
| tree |
In the terminal, display memory usage in a tree structure |
| parse |
Use debug mode to display the memory usage of each line |
| summary |
Summarize memory usage profile during terminal operation |
| stats |
Display memory usage in terminal in great detail |
►►► run command usage
-
python3 -m memray run --help get help
| parameter |
effect |
| -oOUTPU,--output OUTPUT |
Specify where to output results |
| --live |
Start live tracking session mode |
| --live-remote |
Start a live tracking session and wait for the client to connect |
| --live-port LIVE_PORT, -p LIVE_PORT |
The port to use when starting real-time tracing |
| --native |
Tracing the C/C++ stack |
| --follow-fork |
Track allocations in child processes forked by a script |
| --trace-python-allocators |
Record the allocation of pymalloc allocator |
| -q, --quiet |
Does not display any trace-specific output when running |
| -f, --force |
Forced repurchase of existing files |
| --compress-on-exit |
Use lz4 to compress the generated file after tracking is completed |
| --no-compress |
Generated files without lz4 compression |
| -c |
program passed in as a string |
| -m |
Run library modules as scripts |
-
python3 -m memray run xxx.py directly analyzes the memory usage of a certain py file, and a memory usage record file of 'memray-py file name.process id.bin' will be generated under the current path. Of course, you can also follow -o outFiPath to specify the output path. If the py file being run is module code, you can also use the -m xxx.py mode to run it.
The 'memray-py filename.processid.bin' file can be converted into an html-flame graph report by python3 -m memray flamegraph memray-pyfilename.processid.bin
As shown in the figure above, from top to bottom, it shows the calling process of the program. The width represents how much memory the function takes up.
-
python3 -m memray run --native xxxx.py will track and analyze the memory consumed by calling the underlying C/C++ functions in the python code
-
python3 -m memray run --trace-python-allocators xxx.py traces and analyzes the python program memory allocator pymalloc
This seems to have the same effect as without adding parameters, but in fact it is completely different. This method will deeply track memory allocation. There are four common memory allocators in Python (malloc, free, realloc, pymalloc). This parameter is very useful when Python has a memory overflow. However, with this parameter added, the transportation speed will be slower and the files generated by the collected data will be larger.
-
python3 -m memray run --live xxx.py displays traced memory data in live screen mode.
By default, the total memory data is arranged from large to small; press "O" to sort and display the memory objects according to private memory from large to small; press "A" to sort the memory objects according to the number of memory allocations from high to low. Sort.
With this statistical data, you can quickly locate which objects occupy a large amount of memory and which objects are frequently allocated memory. These objects are the key analysis objects.
flamegraph command---generate flame graph report
-
python3 -m memray flamegraph --help Get help
-
python3 -m memray flamegraph xxx.bin generates flame graph
table command--generate table report
-
python3 -m memray table --help Get help
-
python3 -m memray table xxxx.bin converts the bin file into a table report
tree command--generate tree report
-
python3 -m memray tree --help Get help
-
python3 -m memray tree xxxx.bin converts the bin file into a tree report
summary command--generate summary report
-
python3 -m memray summary --help Get help
-
python3 -m memray summary xxxx.bin analyzes the bin file and generates a summary report
stats command---generate detailed statistical reports
-
python3 -m memray stats --help get help
-
python3 -m memray stats xxxx.bin analyzes the bin file and generates a detailed report
Finally: The complete software testing video tutorial below has been compiled and uploaded. Friends who need it can get it by themselves [guaranteed 100% free]
Software Testing Interview Document
We must study to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Alibaba, Tencent, Byte, etc., and some Byte bosses have given authoritative answers. After finishing this set I believe everyone can find a satisfactory job based on the interview information.
