Python performance testing tool

Reprint an article, url: http://blog.csdn.net/xiemanR/article/details/72763234

Usually, the test of the code is basically to test the function. If you want to develop open source code yourself, it is necessary to write test cases and test performance.

timeit

timeit.timeit() only outputs a time in seconds

timeit.repeat() returns a list, each time

2.profile

profile: A performance testing module implemented in pure Python, with the same interface as cProfile.

>>> import profile
>>> def fun():
   for i in range(100000):
      a = i * i

      

>>> profile.run('fun()')
         5 function calls in 0.031 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.016    0.016 :0(exec)
        1    0.016    0.016    0.016    0.016 :0(setprofile)
        1    0.016    0.016    0.016    0.016 <pyshell#13>:1(fun)
        1    0.000    0.000    0.016    0.016 <string>:1(<module>)
        1    0.000    0.000    0.031    0.031 profile:0(fun())
        0    0.000             0.000          profile:0(profiler)


>>>

ncall: the number of times the function is run

tottime: the total running time of the function, minus the running time of calling subfunctions in the function

First percall: percall = tottime/nclall

cumtime: The adjusted running time of the function and all its sub-functions, that is, the time from the beginning of the function call to the end.

Second percall: percall = cumtime/nclall

3.cProfile

profile: The performance test module implemented by C language, the interface is the same as profile.

>>> import cProfile
>>> def fun():
   for i in range(100000):
      a = i * i

      
>>> cProfile.run('fun()')
         4 function calls in 0.024 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.024    0.024    0.024    0.024 <pyshell#17>:1(fun)
        1    0.000    0.000    0.024    0.024 <string>:1(<module>)
        1    0.000    0.000    0.024    0.024 {built-in method exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}


>>>

ncalls, tottime, percall, and cumtime have the same meaning as profile.

4.line_profiler

Install:

pip install line_profiler

After installation, kernprof.py will be added to the environment variables.

line_profiler can count the execution times and execution time of each line of code, and the time unit is subtle.

Test code:

C:\Python34\test.py

import time


@profile
def fun():
    a = 0
b = 0
for i in range(100000):
        a = a + i * i

    for i in range(3):
        b += 1
time.sleep(0.1)

    return a + b


fun()

use:

1. Add @profile decoration to the function that needs to be tested, here we write the test code on the C:\Python34\test.py file.

2. Run the command line: kernprof -l -v C:\Python34\test.py

The output is as follows:

Total Time: The total running time of the test code
Hits: The number of times
each line of code was run Time: The total time each line of code was run
Per Hits: The time that each line of code was run once
% Time: The percentage of time that each line of code was run

5.memory_profiler:

The memory_profiler tool can count the memory size occupied by each line of code.

Install:

pip install memory_profiler

pip install psutil

Test code:

同 line_profiler。

use:

1. Add @profile decoration to the function to be tested

2. Execute the command: python -m memory_profiler C:\Python34\test.py The

output is as follows:

6. PyCharm graphical performance test tool:

PyCharm provides a graphical performance analysis tool. For usage, see Using PyCharm's Profile Tool for Python Performance Analysis .

7.objgraph：

objgraph is a utility module that lists objects currently in memory and can be used to locate memory leaks.

objgraph needs to be installed:

pip install objgraph

The method of use is not described here, and Baidu is used by itself.

------------------------------------------- I am the dividing line--- --------------------------------------

The graphical tool of pycharm looks good and can generate call graphs, but this feature is not seen on the community edition. I have tried to generate it with doxygen+graphviz, but only the inheritance graph of the class is the same as the document api, and it will be less than the real inheritance relationship in the code.

There is a benchmark code for testing website performance with ab in tornado's demo, which needs to be installed first: sudo apt-get install apache2-utils.