Calculating the running time of a program in Python - timeit module

introduction

time.time()方法And time.clock()方法can be used to calculate program execution time and cpu time. However, many times we only want to count the execution time of some code fragments or algorithms. At this time, it is more convenient to use the timeit module.

The timeit module is a module in the Python standard library, which can be used directly without installation. The timeit module is a Python built-in module for counting the execution time of small pieces of code, and it also provides a command line calling interface.

When importing directly import timeit, you can use timeit() 函数and repeat() 函数, as well as the Timer class. When using from timeit import ..., only the Timer class can be imported (limited by global variables all ).

First, the basic usage of timeit

1.1 timeit.timeit() function: Create a Timer instance and run the code for timing. By default, the code will be executed one million times.

Basic syntax:

timeit.timeit(stmt='pass', setup='pass', timer=<default timer>, number=1000000)

Parameter Description:

• stmt : Pass in the code that needs to be tested. You can pass in a code expression or a single variable directly, or you can pass in a function. When passing in a function, add parentheses after the function name to let the function execute, such as stmt = 'func()'.
• setup : Pass in the operating environment of stmt, such as parameters and variables used in stmt, modules to be imported, etc., eg setup = ‘from __main__ import func' (__main__表示当前的文件). You can write one line of statements, or you can write multiple lines of statements, separated by semicolons when writing multiple lines of statements. .
• timer: It is the basic time unit of the current operating system. By default, it will be automatically obtained according to the operating system of the current operating environment (already defined in the source code), just keep the default.
• number: The number of times to test the code, the default is 1000000 (one million) times, you can modify the number of times to run.
• globals: is the namespace for execution.

timeit.default_timer(): The default timer, which is time.perf_counter()

class timeit.Timer(stmt='pass', setup='pass', timer=, globals=None): A timing class for code execution speed tests. This class has four methods:

• timeit(number=1000000)
• autorange(callback=None)
• repeat(repeat=3, number=1000000)
• print_exc(file=None)

Remarks: The default values ​​of stmt parameter and setup parameter are both pass. If no value is passed, the meaning of testing will be lost, so these two parameters are necessary.

1.2 timeit.repeat() function: Execute the timeit method for the specified number of repetitions, and return a list of results.

Basic syntax:

timeit.repeat(stmt='pass', setup='pass', timer=<default timer>, repeat=3, number=1000000)

Parameter Description:

• repeat : Indicates that the test should be repeated several times, which can be understood as repeatedly executing the timeit() function with the same parameters. The final result is returned as a list, and the repeat is 3 times by default.

2. Execute the timeit command in the command line command to calculate the running time of the program

Command line interface:

When invoking the program from the command line, the following forms are used:

python -m timeit [-n N] [-r N] [-s S] [-t] [-c] [-h] [statement ...]

Parameter Description:

• -n N, --number=N: number of executions;
• -r N, --repeat=N: number of timer repeats (default 3);
• -s S, --setup=S: Execution environment configuration (usually this statement is only executed once, default pass);
• -t, --time : use time.time() (default on all platforms except Windows);
• -c, --clock: use time.clock() (default on Windows);
• -v, --verbose: print raw time results, get more numerical precision;
• -h, --help: help;

(1)计算输出形如：'0_1_2...'的时间
python3 -m timeit '"_".join(str(n) for n in range(100))'
>>>
20000 loops, best of 5: 18.5 usec per loop
(2)计算输出形如：'0_1_2...'的时间
python3 -m timeit '"-".join([str(n) for n in range(100)])'
>>>
20000 loops, best of 5: 16.4 usec per loop
(3)计算输出形如：'0_1_2...'的时间
python3 -m timeit '"_".join(map(str, range(100)))'
>>>
20000 loops, best of 5: 13.3 usec per loop

python -m timeit -s 'text = "sample string"; char = "g"'  'char in text'
>>>
10000000 loops, best of 5: 27.5 nsec per loop


import timeit
t = timeit.Timer('char in text', setup='text = "sample string"; char = "g"')
t.timeit()
>>>
0.060282987000391586

3. Call the timeit module inside the Python program to calculate the running time of the program

3.1. Test the execution time of single/multi-line statements:

# 测试单行语句执行时间
import timeit
timeit.timeit('"_".join(map(str, range(100)))',number=10)
>>>
0.0002646529974299483
timeit.timeit('"_".join(str(n) for n in range(100))',number=10)
>>>
0.0005391980012063868
timeit.timeit('"_".join([str(n) for n in range(100)])',number=10)
>>>
0.000366011998266913

# 测试多行语句执行时间
import timeit
s = """\
try:
    int.__bool__
except AttributeError:
    pass
"""
timeit.timeit(stmt=s, number=100000)

3.2. Specify the calling object through the setup parameter in the method:

示例1：
def test():
    """Stupid test function"""
    L = []
    for i in range(100):
        L.append(i)

if __name__ == '__main__':
    import timeit
    print(timeit.timeit("test()", setup="from __main__ import test"))

>>>
6.578784502999952

## 或者
def test():
	"""Stupid test function"""
    L = [i for i in range(100)]

if __name__ == '__main__':
    import timeit
	print(timeit.timeit("test()", setup="from __main__ import test"))
>>>
3.24046479000026

示例2：
def matrix():
    """矩阵乘法运算"""
    import torch
    M = torch.rand(10, 10)
    M.mm(M).mm(M)
if __name__ == '__main__':
    import timeit
    print(timeit.timeit("matrix()",number=3, setup="from __main__ import matrix"))
>>>
0.001695301000154359

3.3. Specify the running space through the globals parameter:

def f(x):
    return x**2
def g(x):
    return x**4
def h(x):
    return x**8

import timeit
print(timeit.timeit('[func(42) for func in (f,g,h)]', globals=globals()))
>>>
1.3725472270016326

Reference link: timeit

Full example:

# coding=utf-8
def insert_time_test():
    insert_list = list()
    for i in range(10):
        insert_list.insert(0, i)
  

def append_time_test():
    append_list = list()
    for i in range(10):
        append_list.append(i)
 
if __name__ == '__main__':
    import timeit

insert_time_timeit = timeit.timeit(stmt='insert_time_test()', 
                                   setup='from __main__ import insert_time_test')
print('insert_time_timeit: ', insert_time_timeit)
append_time_timeit = timeit.timeit(stmt='append_time_test()', 
                                   setup='from __main__ import append_time_test')
print('append_time_timeit: ', append_time_timeit)
>>>
insert_time_timeit:  1.5472285019968695
append_time_timeit:  1.0619552210009715


insert_time_timeit = timeit.timeit(stmt='list(insert_list.insert(0, i) for i in init_list)',
                                   setup='insert_list=list();init_list=range(10)',
                                   number=100000)
print('insert_time_timeit: ', insert_time_timeit)
append_time_timeit = timeit.timeit(stmt='list(append_list.append(i) for i in init_list)',
                                   setup='append_list=list();init_list=range(10)',
                                   number=100000)
print('append_time_timeit: ', append_time_timeit)
>>>
insert_time_timeit:  416.30776414700085
append_time_timeit:  0.3249605919991154


insert_time_repeat = timeit.repeat(stmt='insert_time_test()',
                                   setup='from __main__ import insert_time_test')
print('insert_time_repeat: ', insert_time_repeat)
append_time_repeat = timeit.repeat(stmt='append_time_test()',
                                   setup='from __main__ import append_time_test')
print('append_time_repeat: ', append_time_repeat)
>>>
insert_time_repeat:  [1.5643876249996538, 1.573867484999937, 1.5350732170008996, 1.4930691439985821, 1.613208124999801]
append_time_repeat:  [1.2118435169977602, 1.1756933159995242, 1.2724871930004156, 1.27145235900025, 1.3099699200029136]

When using the timeit module, you can use timeit.timeit(), tiemit.repeat() directly, or use timeit.Timer() to generate a Timer object first, and then use the timeit() and repeat() functions with the TImer object , which is more flexible.

1. Use timeit.timeit() and tiemit.repeat() directly:

import timeit 

print(timeit.timeit(stmt= 'list(i**2 for i in normal_list)',setup = 'normal_list=range(10000)',number=10))
#0.3437936799875755
print(timeit.repeat(stmt= 'list(i**2 for i in normal_list)', setup='normal_list=range(10000)',repeat=2,number=10))
#[0.33649995761778984, 0.3394490767789293]
#setup 为复合语句
print(timeit.timeit(stmt= 'list(i**2 for i in normal_list)',setup = 'a=10000;normal_list=range(a)',number=10))
#0.33272367424748817
print(timeit.repeat(stmt= 'list(i**2 for i in normal_list)', setup='a=10000;normal_list=range(a)',repeat=2,number=10))
#[0.3323106610316342, 0.3356380911962764]

def func():
    normal_list=range(10000)
    L = [i**2 for i in normal_list]

#stmt为函数
print(timeit.timeit("func()", setup="from __main__ import func",number=10))
#0.12436874684622312
print(timeit.repeat("func()", setup="from __main__ import func",repeat=2,number=10))
#[0.12142133435126468, 0.12079555675148601]

It is faster to use the function directly.

2. Generate a Timer first, and then call timeit() and repeat():

import timeit 

#生成timer
timer1 = timeit.Timer(stmt= 'list(i**2 for i in normal_list)',setup = 'normal_list=range(10000)')
#调用timeit和repeat时还传number和repeat参数
print(timer1.timeit(number=10))
#0.34721554568091145
print(timer1.repeat(repeat=2,number=10))
#[0.3391925079630199, 0.34103400077255097]

#setup 为复合语句
timer1 = timeit.Timer(stmt= 'list(i**2 for i in normal_list)',setup = 'a=10000;normal_list=range(a)')
print(timer1.timeit(number=10))
0.34383463997592467
print(timer1.repeat(repeat=2,number=10))
#[0.34573984832288773, 0.34413273766891006]

#stmt为函数
def func():
    normal_list=range(10000)
    L = [i**2 for i in normal_list]

timer1 = timeit.Timer("func()", setup="from __main__ import func")
print(timer1.timeit(number=10))
#0.1223264363160359
print(timer1.repeat(repeat=2,number=10))
#[0.12266321844246209, 0.1264150395975001]

For details about the timeit module in the official Python tutorial, please refer to the documentation:

http://docs.python.org/library/timeit.html

https://docs.python.org/zh-cn/3/library/timeit.html

4. Appendix: common usage

time.clock() calculates the CPU time, and the accuracy is relatively high on the windows platform
. time.time() calculates the running time of the program, which will be affected by the machine load. The accuracy is relatively high on platforms other than windows.

So we can use different methods according to the platform.

# 选择不同运行平台下的计数器
import time
import sys

if sys.platform == "win32":
    # On Windows, the best timer is time.clock()
    default_timer = time.clock
else:
    # On most other platforms the best timer is time.time()
    default_timer = time.time

import timeit
# 通过开始、结束的时间差，得到程序某函数段的运行时间
b = timeit.default_timer()
# to some
e = timeit.default_timer()
print(e - b)
>>>
7.159800225053914e-05

Reference link: Python timer timeit