I've come across some strange behaviour trying to time python scripts. Minimum example:
foobar.py:
foo = 'Hello'
print(''.join(c for c in foo if c not in 'World'))
print(''.join(c for c in 'World' if c not in foo))
timer.py:
import timeit
timeit.repeat(stmt="exec(open('foobar.py').read())", repeat=1, number=1)
When I run foobar.py, I get the expected output:
> python3 foobar.py
He
Wrd
However, when I run timer.py, I get the following error:
> python3 timer.py
He
Traceback (most recent call last):
File "timer.py", line 2, in <module>
timeit.repeat(stmt="exec(open('foobar.py').read())", repeat=1, number=1)
File "/usr/lib/python3.7/timeit.py", line 237, in repeat
return Timer(stmt, setup, timer, globals).repeat(repeat, number)
File "/usr/lib/python3.7/timeit.py", line 204, in repeat
t = self.timeit(number)
File "/usr/lib/python3.7/timeit.py", line 176, in timeit
timing = self.inner(it, self.timer)
File "<timeit-src>", line 6, in inner
File "<string>", line 3, in <module>
File "<string>", line 3, in <genexpr>
NameError: name 'foo' is not defined
Perhaps the most bizzare thing about it is that the first print statement in foobar.py works fine, while the second one does not. Executing foobar.py using exec without timeit wrapper works fine as well.
Anyone have an explanation for this odd behaviour?
This is actually not limited to timeit in combination with exec, but an issue with exec alone: The statements are being executed in a local namespace and the generator inside str.join uses another (new) local namespace, where the previously set foo isn't known.
Class definition blocks and arguments to
exec()andeval()are special in the context of name resolution. A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution with an exception that unbound local variables are looked up in the global namespace. The namespace of the class definition becomes the attribute dictionary of the class. The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods – this includes comprehensions and generator expressions since they are implemented using a function scope. This means that the following will fail:class A: a = 42 b = list(a + i for i in range(10))
Source: https://docs.python.org/3/reference/executionmodel.html#resolution-of-names
Also see list comprehension in exec with empty locals: NameError for example.
As a fix, you can set the globals dictionary with the second argument of exec, so all statements use the same dictionary:
timeit.repeat(stmt="exec(open('foobar.py').read(), locals())", repeat=1, number=1)
Or you could just drop exec altogether and use import:
timeit.repeat(stmt="import foobar", repeat=1, number=1)