1 Misusing expressions as default values for function arguments
Python allows specifying function parameters by providing default values for the function, but when the default value is mutable, some problems arise:
def foo(bar=[]):
bar.append('baz')
return bar
In the code above, the expectation is that foo()repeated calls (i.e. without specifying the bar parameter) will always return 'baz', so it is assumed that each foo()call baris set to [].
However, let's take a look at what actually happens when this operation is performed:
>>> foo()
['baz']
>>> foo()
['baz', 'baz']
Hey, why does the default value append 'baz'to the existing list every time it's called, instead of creating a new list every time?
The answer is: the default value of a function parameter is only calculated once when the function is defined . Thus parameters are baronly foo()initialized to their default values when first defined, but subsequently invoked foo()(i.e. without barparameters specified), will continue to use barthe same list that was initially initialized with.
FYI, a common workaround is as follows:
def foo(bar=None):
if not bar:
bar = []
bar.append('baz')
return bar
2 Wrong use of class variables
Consider an example:
>>> class A(object):
... x = 1
...
>>> class B(A):
... pass
...
>>> class C(A):
... pass
...
>>> print(A.x, B.x, C.x)
1 1 1
There is no problem with the above output, please continue to read:
>>> B.x = 2
>>> print(A.x, B.x, C.x)
1 2 1
The output is still as expected, so next:
>>> A.x = 3
>>> print(A.x, B.x, C.x)
Consider the output above:
3 2 3
What's happening here? We only changed Ax, why did Cx also change?
In python, class variables are handled internally as dictionaries and follow what is often called the Method Resolution Order (MRO), so in the code above, since the attribute is not found in , it will be looked up in its base Cclass x. Numbers, in other words, Chave no properties of their own x, so references to Cx are actually worth Ax.
3 Incorrectly specifying parameters for exception blocks
If you use the following code:
# 这段代码是python 2.7版本的
>>> try:
... l = ["a", "b"]
... int(l[2])
... except (ValueError, IndexError): # To catch both exceptions, right?
... pass
...
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
IndexError: list index out of range
The problem here is that the except statement does not take a list of exceptions specified this way, instead, in python2.x the syntax is except Exception, eused to bind exceptions to a specified optional second argument (e in this case) for further inspection. It turns out that in the above code, the IndexError exception is not exceptcaught by the statement, instead, the exception ends up being bound to a IndexErrorparameter named .
In excepta statement, the correct way to catch multiple exceptions is to specify the first argument as the tuple containing all exceptions to catch, and for maximum portability, use the keyword, since both Python2 and Python3 support this syntax as.
>>> try:
... l = ['a', 'b']
... int(l[2])
... except (ValueError, IndexError) as e:
... pass
...
>>>
4 Misinterpreting the Python range specification
Python range resolution is based on the so-called LEGB rules. There are some subtleties in how Python works, let's look at common more advanced Python programming questions:
>>> x = 10
>>> def foo():
... x += 1
... print(x)
...
>>> foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in foo
UnboundLocalError: local variable 'x' referenced before assignment
The above problem arises because when you assign a value to a variable in a scope, Python automatically treats the variable as local to that scope and hides any similarly named variables in any outer scopes.
But when using a list, there is a special phenomenon, please see the following code example:
>>> lst = [1, 2, 3]
>>> def foo1():
... lst.append(5)
...
>>> foo1()
>>> lst
[1, 2, 3, 5]
>>> lst = [1, 2, 3]
>>> def foo2():
... lst += [5]
...
>>> foo2()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in foo2
UnboundLocalError: local variable 'lst' referenced before assignment
Huh? Why does foo1 run fine, but foo2 fails? ?
The answer is the same as the previous example question, but certainly more subtle. foo1Instead of assigning a value to lst, foo2 does. Remembering the shorthand lst += [5]for actual lst = lst + [5], we see that foo2 is assigning a value to lst, so Python assumes it's in local scope. But the value lst we are assigning is lst itself, so it is undefined.
5 Modifying a list while iterating
The problem with the following code is fairly obvious:
>>> odd = lambda x: bool(x % 2)
>>> numbers = [n for n in range(10)]
>>> for i in range(len(numbers)):
... if odd(numbers[i]):
... del numbers[i]
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
IndexError: list index out of range
Removing items from a list or array while iterating is a common Python problem. Fortunately Python incorporates many elegant programming paradigms that can simplify code if used properly. An added benefit is that the simpler code is less likely to cause iteration problems due to accidental removal of list items. It works perfectly:
>>> odd = lambda x : bool(x % 2)
>>> numbers = [n for n in range(10)]
>>> numbers[:] = [n for n in numbers if not odd(n)] # ahh, the beauty of it all
>>> numbers
[0, 2, 4, 6, 8]
6 Confused about how Python binds variables in closures
Consider the following examples:
>>> def create_multipliers():
... return [lambda x: i * x for i in range(5)]
...
>>> for multiplier in create_multipliers():
... print(multiplier(2))
...
You might expect the following output:
0
2
4
6
8
But what you get is:
8
8
8
8
8
This is because when Python calls the inner function, the variable value used in the closure is caused by the late binding behavior. So in the code above, whenever any of the returned functions are called, at the time i is called it is looked up in the surrounding scope for the value, that's when the loop has completed and thus i has been assigned its final value of 4.
The solution to this common problem is a bit of a hack:
>>> def create_multipliers():
... return [lambda x, i=i : i * x for i in range(5)]
...
>>> for multiplier in create_multipliers():
... print(multiplier(2))
...
0
2
4
6
8
>>>
Here default parameters are exploited to generate anonymous functions to achieve the desired behavior which some will call elegant, some will find negligible, and some will hate it. But as a Python developer, you have to understand it anyway.
7 Create circular references
Suppose you have two files, a.pyand b.pyand each imports the other, like so:
In a.py:
import b
def f():
return b.x
print(f())
In b.py:
import a
x = 1
def g():
print(a.f())
First let's try to importa.py
>>> import a
1
So far, no exceptions have occurred, maybe this will bring you a surprise, after all, we have a circular import problem here, probably it should be a problem, shouldn't it? The answer is that the mere existence of circular imports is not a problem of Python itself. If a module is already imported, Python is smart enough not to try to re-import it. But depending on each module trying to access a function or variable defined in another module, you might run into some problems.
So back to the example, when we import , is there a problem with a.pyit importing ? b.pySince the b.pydoes not need to import any variables a.pyfrom it , since the only thing called is still called a.f()at call time, nothing is called at this time or in , so everything looks good.g()a.pyb.pyg()
b.pyWhat happens if we try to import ? (provided that it is not imported first a.py)
>>> import b
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/wen/b.py", line 1, in <module>
import a
File "/home/wen/a.py", line 8, in <module>
print(f())
File "/home/wen/a.py", line 5, in f
return b.x
AttributeError: module 'b' has no attribute 'x'
>>>
Here comes the problem. In the import b.py, he tries to import a.py, which in turn calls f()what is trying to access b.x, but b.xit hasn't been defined, so there is AttributeErrora problem.
Here is a simple solution to deal with this problem, just modify b.pyand g()import in a.py:
x = 1
def g():
import a
print(a.f())
When we import it, everything will be fine:
>>> import b
>>> b.g()
1
1
8 Name conflicts with Python standard library modules
One of the strengths of Python is that it provides a rich set of library modules "out of the box". But if you avoid it consciously, the chances of a custom module conflicting with the Python standard library module will increase a lot.
9 Failed to resolve differences between Python2 and Python3
consider the filefoo.py
import sys
def bar(i):
if i == 1:
raise KeyError(1)
if i == 2:
raise ValueError(2)
def bad():
e = None
try:
bar(int(sys.argv[1]))
except KeyError as e:
print('key error')
except ValueError as e:
print('value error')
print(e)
bad()
On Python2, it works fine:
$ python foo.py 1
key error
1
$ python foo.py 2
value error
2
But on Python3:
$ python3 foo.py 1
key error
Traceback (most recent call last):
File "foo.py", line 19, in <module>
bad()
File "foo.py", line 17, in bad
print(e)
UnboundLocalError: local variable 'e' referenced before assignment
The "problem" is that in Python 3 the exception object is not accessible outside the scope of the except block. (The reason is that otherwise, it keeps a reference loop of the stack frame in memory until the garbage collector runs and clears the reference from memory.
One way to avoid this problem is to maintain a reference to the exception object outside the scope of the block, keeping it accessible except. Here's a version of the previous example that uses this technique, resulting in Python 2 and Python 3 compatible code:
import sys
def bar(i):
if i == 1:
raise KeyError(1)
if i == 2:
raise ValueError(2)
def good():
exception = None
try:
bar(int(sys.argv[1]))
except KeyError as e:
exception = e
print('key error')
except ValueError as e:
exception = e
print('value error')
print(exception)
good()
Running on Python3:
$ python3 foo.py 1
key error
1
$ python3 foo.py 2
value error
2
10 __del__Methods of Abuse
Say you have this in a file called mod.py:
import foo
class Bar(object):
...
def __del__(self):
foo.cleanup(slef.myhandle())
Then you try to do this another_mod.py:
import mod
mybar = mod.Bar()
You will get an ugly one AttributeError.
When the interpreter is closed, the module's global variables are all set to None. So in the example above, where the __del__call is made, the name foois already set to None.
The solution is to use atexit.register(). This way, when your program finishes executing (on normal exit), your registered handler will start before the interpreter shuts down:
import foo
import atexit
def cleanup(handle):
foo.cleanup(handle)
class Bar(object):
def __init__(self):
...
atexit.register(cleanup, self.myhandle)
This implementation provides a clean and reliable way to call any required cleanup functions on normal program termination. Obviously it's up to foo.cleanup to decide what to do with the object bound to the name self.myhandle , but you get the idea.