Article directory
Bugs, Debugging and Testing
During the running of the program, various errors will always be encountered.
Some errors are caused by problems in program writing. For example, a string should be output instead of an integer. This kind of error is usually called a bug , and bugs must be fixed.
Some errors are caused by user input. For example, asking the user to enter an email address results in an empty string. This error can be handled accordingly by checking the user input.
There is another type of error that is completely unpredictable during the running of the program. For example, when writing a file, the disk is full and cannot be written, or the network is suddenly disconnected when data is captured from the network. This type of error is also called an exception and must usually be handled in the program. Otherwise, the program will terminate and exit due to various problems.
Python has a built-in exception handling mechanism to help us handle errors.
In addition, we also need to track the execution of the program to see whether the values of variables are correct. This process is called debugging . Python's pdb allows us to execute code in a single-step manner.
Finally, writing tests is also important. With good tests, we can run them repeatedly after the program is modified to ensure that the program output matches the tests we wrote.
Error handling
try
Let's look at trythe mechanism using an example:
try:
print('try...')
r = 10 / 0
print('result:', r)
except ZeroDivisionError as e:
print('except:', e)
finally:
print('finally...')
print('END')
When we think that some code may go wrong, we can tryuse it to run this code. If there is an error in execution, the subsequent code will not continue to execute, but will jump directly to the error handling code, that is, exceptthe statement block. exceptAfter , If there is a statement block, the statement block finallyis executed . At this point, the execution is completed.finally
The above code 10 / 0will generate a division operation error when calculating:
try... # 后面没有继续输出 result...
except: division by zero
finally...
END
If the divisor 0is changed to 2, the execution result is as follows:
try...
result: 5.0
finally...
END
Since no error occurs, exceptthe statement block will not be executed, but finallyif there is, it will definitely be executed (no finallystatements are allowed).
There can be multiple exceptto catch different types of errors.
Here one exceptcaptures ValueErrorand the other exceptcaptures ZeroDivisionError.
If no error occurs, you can exceptadd one after the statement block else. When no error occurs, elsethe statement will be automatically executed.
try:
print('try...')
r = 10 / int('2')
print('result:', r)
except ValueError as e:
print('ValueError:', e)
except ZeroDivisionError as e:
print('ZeroDivisionError:', e)
else:
print('no error!')
finally:
print('finally...')
print('END')
The result is as follows:
try...
result: 5.0
no error!
finally...
END
Python's errors are actually classes, and all error types are inherited from them BaseException. Therefore except, what you need to pay attention to when using it is that it not only captures errors of this type, but also "catches all" its subclasses. for example:
try:
foo()
except ValueError as e:
print('ValueError')
except UnicodeError as e:
print('UnicodeError')
The second one exceptcan never be captured UnicodeError, because if UnicodeErrorit is ValueErrora subclass, it is also exceptcaptured by the first one.
All Python errors are BaseExceptionderived from classes. See here for common error types and inheritance relationships:
https://docs.python.org/3/library/exceptions.html#exception-hierarchy
try...exceptAnother huge benefit of using captured errors is that it can span multiple layers of calls, such as function main()calls bar(), bar()calls foo(), and an foo()error occurs. At this time, as long as main()it is captured, it can be processed:
def foo(s):
return 10 / int(s)
def bar(s):
return foo(s) * 2
def main():
try:
bar('0')
except Exception as e:
print('Error:', e)
finally:
print('finally...')
if __name__ == '__main__':
main()
# 结果如下：
# Error: division by zero
# finally...
In other words, you don't need to catch errors in every possible place that can go wrong, you just need to catch errors at the appropriate level. In this way, the trouble of writing is greatly reduced try...except...finally.
call stack
If the error is not caught, it will keep throwing up, and finally it will be caught by the Python interpreter, an error message will be printed, and then the program will exit. Let’s take a look err.py:
$ python3 err.py
Traceback (most recent call last):
File "err.py", line 11, in <module>
main()
File "err.py", line 9, in main
bar('0')
File "err.py", line 6, in bar
return foo(s) * 2
File "err.py", line 3, in foo
return 10 / int(s)
ZeroDivisionError: division by zero
When an error occurs, you must analyze the error call stack information to locate the error location.
Log errors
If the error is not caught, the Python interpreter can naturally print out the error stack, but the program will also be terminated. Now that we can capture the error, we can print out the error stack, analyze the cause of the error, and at the same time, let the program continue to execute.
Python’s built-in loggingmodule makes logging error messages very easy:
import logging
def foo(s):
return 10 / int(s)
def bar(s):
return foo(s) * 2
def main():
try:
bar('0')
except Exception as e:
logging.exception(e)
main()
print('END')
The same error occurs, but the program will continue execution after printing the error message and exit normally:
ERROR:root:division by zero
Traceback (most recent call last):
File "E:/csdn/9_错误处理.py", line 49, in main
bar('0')
File "E:/csdn/9_错误处理.py", line 45, in bar
return foo(s) * 2
File "E:/csdn/9_错误处理.py", line 42, in foo
return 10 / int(s)
ZeroDivisionError: division by zero
END
Through configuration, loggingerrors can also be recorded in the log file to facilitate subsequent troubleshooting.
throw error
Because errors are classes, catching an error is catching an instance of that class. Therefore, errors do not arise out of thin air, but are intentionally created and thrown. Python's built-in functions can throw many types of errors, and functions we write ourselves can also throw errors.
If you want to throw an error, you can first define an error class as needed, select the inheritance relationship, and then use a raisestatement to throw an error instance :
class FooError(ValueError):
pass
def foo(s):
n = int(s)
if n == 0:
raise FooError('invaild value: %s' % s)
return 10 / n
print(foo('0'))
Execution, we can finally trace our own defined errors FooError:
Traceback (most recent call last):
File "E:/csdn/9_错误处理.py", line 70, in <module>
print(foo('0'))
File "E:/csdn/9_错误处理.py", line 66, in foo
raise FooError('invaild value: %s' % s)
__main__.FooError: invaild value: 0
Define our own error types only when necessary. If you can choose Python's built-in error types (for example ValueError, TypeError), try to use Python's built-in error types .
Finally, let's look at another way of error handling:
def foo(s):
n = int(s)
if n==0:
raise ValueError('invalid value: %s' % s)
return 10 / n
def bar():
try:
foo('0')
except ValueError as e:
print('ValueError!')
raise
bar()
In bar()the function, we have obviously caught the error, but ValueError!after printing one, we throw the error raiseout through the statement. Isn't this unhealthy?
In fact, this error handling method is not only harmless, but also quite common. The purpose of capturing errors is just to record them for subsequent tracking. However, since the current function does not know how to handle the error, the most appropriate way is to continue to throw it up and let the top-level caller handle it. For example, when an employee cannot handle a problem, he will throw the problem to his boss. If his boss cannot handle it, he will keep throwing it up, and eventually it will be thrown to the CEO to handle.
raiseIf the statement does not take parameters, the current error will be thrown unchanged. In addition, in exceptan raiseError, you can also convert one type of error into another type:
try:
10 / 0
except ZeroDivisionError:
raise ValueError('input error!')
The result is as follows:
Traceback (most recent call last):
File "E:/csdn/9_错误处理.py", line 93, in <module>
10 / 0
ZeroDivisionError: division by zero
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:/csdn/9_错误处理.py", line 95, in <module>
raise ValueError('input error!')
ValueError: input error!
As long as it is a reasonable conversion logic, it should never be IOErrorconverted into something irrelevant ValueError.
practise
from functools import reduce
def str2num(s):
try:
number = int(s)
except:
try:
number = float(s)
except:
print("please input a number!!!")
finally:
return number
def calc(exp):
ss = exp.split('+')
ns = map(str2num, ss)
return reduce(lambda acc, x: acc + x, ns)
def main():
r = calc('100 + 200 + 345')
print('100 + 200 + 345 =', r)
r = calc('99 + 88 + 7.6')
print('99 + 88 + 7.6 =', r)
r = calc('100 + s')
main()
debug
A complete set of debugging methods is needed to fix bugs.
The first method is simple, direct, crude and effective, which is to print()print out the variables that may have problems.
The biggest disadvantage of using print()it is that you have to delete it in the future. Think about it, it is everywhere in the program print(), and the running results will also contain a lot of junk information. So, we have a second method: assertions.
affirmation
Wherever it print()is used to assist viewing, assertions can be used instead:
def foo(s):
n = int(s)
assert n != 0, 'n is zero!'
return 10 / n
print(foo('9'))
assertIt means that the expression n != 0should be True, otherwise, according to the logic of program operation, the following code will definitely go wrong.
If the assertion fails, assertthe statement itself throws AssertionError:
Traceback (most recent call last):
File "E:/csdn/9_错误处理.py", line 135, in <module>
print(foo('0'))
File "E:/csdn/9_错误处理.py", line 131, in foo
assert n != 0, 'n is zero!'
AssertionError: n is zero!
If the program is full of them assert, it print()will be no better than. However, the Python interpreter can be -Oturned off with parameters when starting it assert:
PS E:\csdn> python -O 9_错误处理.py # 这里是英文大写O！！！！
Traceback (most recent call last):
File "9_错误处理.py", line 135, in <module>
print(foo('0'))
File "9_错误处理.py", line 132, in foo
return 10 / n
ZeroDivisionError: division by zero
After closing, you can view all assertstatements as pass.
logging
The third way is to print()replace with logging. And assertthan, loggingno error will be thrown, and can be output to a file:
import logging
logging.basicConfig(level=logging.INFO)
s = '0'
n = int(s)
logging.info('n = %d' % n)
print(10 / n)
The result is as follows:
INFO:root:n = 0
Traceback (most recent call last):
File "E:/csdn/9_错误处理.py", line 145, in <module>
print(10 / n)
ZeroDivisionError: division by zero
This is loggingthe benefit, it allows you to specify the level of recording information, there are several levels, such as debug, info, warning, etc., when we specify it , it will not work. In the same way, once specified, the sum will not work. In this way, you can safely output different levels of information without deleting it. Finally, you can uniformly control which level of information is output.errorlevel=INFOlogging.debuglevel=WARNINGdebuginfo
loggingAnother benefit is that through simple configuration, a statement can be output to different places at the same time, such as console and file. (I haven’t learned how to use it yet, but the summary says: logging is the ultimate weapon!)
unit test
"Test-Driven Development" (TDD: Test-Driven Development).
Putting test cases into a test module is a complete unit test.
If the unit test passes, it means that the function we tested can work normally. If the unit test fails, either there is a bug in the function, or the test conditions are entered incorrectly. In short, it needs to be fixed to make the unit test pass.
The biggest benefit of this test-driven development model is to ensure that the behavior of a program module conforms to the test cases we design. When modified in the future, it can be greatly guaranteed that the module behavior will still be correct.
Let's write a Dictclass that behaves the dictsame way but can be accessed through properties, like this:
>>> d = Dict(a=1, b=2)
>>> d['a']
1
>>> d.a
1
mydict.pycode show as below:
class Dict(dict):
def __init__(self, **kwargs):
super(Dict, self).__init__(**kwargs)
def __getattr__(self, key):
try:
return self[key]
except KeyError:
raise AttributeError(r"'Dict' object has no attribute '%s'" % key)
def __setattr__(self, key, value):
self[key] = value
In order to write unit tests, we need to introduce the module that comes with Python unittestand write it mydict_test.pyas follows:
import unittest
from mydict import Dict
class TestDict(unittest.TestCase): # 编写测试类，从unittest.TestCase继承
def test_init(self):
d = Dict(a=1, b='test')
self.assertEqual(d.a, 1)
self.assertEqual(d.b, 'test')
self.assertTrue(isinstance(d, dict))
def test_key(self):
d = Dict()
d['key'] = 'value' # 第一种访问方式
self.assertTrue('key' in d)
self.assertEqual(d['key'], 'value')
def test_attr(self):
d = Dict()
d.key = 'value' # 第二种访问方式
self.assertTrue('key' in d)
self.assertEqual(d['key'], 'value')
def test_keyerror(self):
d = Dict()
with self.assertRaises(KeyError):
value = d['empty']
def test_attrerror(self):
d = Dict()
with self.assertRaises(AttributeError):
value = d.empty
When writing unit tests, we need to write a test class that unittest.TestCaseinherits from .
Methods that begin with testare test methods. testMethods that do not begin with are not considered test methods and will not be executed during testing.
For each type of test you need to write a test_xxx()method. Since unittest.TestCasemany built-in conditional judgments are provided, we only need to call these methods to assert whether the output is what we expect. The most commonly used assertions are assertEqual():
self.assertEqual(abs(-1), 1) # 断言函数返回的结果与1相等
Another important assertion is to expect an Error of a specified type to be thrown . For example, d['empty']when accessing a non-existent key, the assertion will throw KeyError:
with self.assertRaises(KeyError):
value = d['empty']
When d.emptyaccessing a non-existent key, we expect to throw AttributeError:
with self.assertRaises(AttributeError):
value = d.empty
Run unit tests
Once the unit tests are written, we can run the unit tests. The simplest way to run it is mydict_test.pyto add two lines of code at the end:
if __name__ == '__main__':
unittest.main()
This can be mydict_test.pyrun as a normal python script:
$ python mydict_test.py
Another way is -m unittestto run the unit tests directly via arguments on the command line:
$ python -m unittest mydict_test
.....
----------------------------------------------------------------------
Ran 5 tests in 0.000s
OK
This is recommended because you can run many unit tests in batches at once, and there are many tools that can run these unit tests automatically.
setUp and tearDown
setUp()There are two special and methods that can be written in unit tests tearDown(). These two methods will be executed before and after each test method is called.
setUp()What is the use of and tearDown()method? Imagine that your test needs to start a database. At this time, you can connect to the database in setUp()the method and tearDown()close the database in the method . This way, you do not have to repeat the same code in each test method:
class TestDict(unittest.TestCase):
def setUp(self):
print('setUp...')
def tearDown(self):
print('tearDown...')
setUp...Run the test again and see that the sum is printed before and after each test method call tearDown....
The result is as follows:
PS E:\csdn> python mydict_test.py
1
1
setUp...
tearDown...
.setUp...
tearDown...
.setUp...
tearDown...
.setUp...
tearDown...
.setUp...
tearDown...
.
----------------------------------------------------------------------
Ran 5 tests in 0.002s
OK
practise
The code after modifying the Student class is as follows, which can make the test pass:
class Student(object):
def __init__(self, name, score):
self.name = name
self.score = score
def get_grade(self):
if self.score > 100 or self.score < 0:
raise ValueError
elif self.score >= 80:
return 'A'
elif self.score >= 60:
return 'B'
else:
return 'C'
Document testing
Python's built-in "doctest" module can directly extract the code in the comments and execute the test.
doctest strictly follows the input and output of the Python interactive command line to determine whether the test results are correct. Only when testing anomalies, you can use it ...to represent a large section of annoying output in the middle.
Let's use doctest to test the class we wrote last time Dict:
# mydict2.py
class Dict(dict):
'''
Simple dict but also support access as x.y style.
>>> d1 = Dict()
>>> d1['x'] = 100
>>> d1.x
100
>>> d1.y = 200
>>> d1['y']
200
>>> d2 = Dict(a=1, b=2, c='3')
>>> d2.c
'3'
>>> d2['empty']
Traceback (most recent call last):
...
KeyError: 'empty'
>>> d2.empty
Traceback (most recent call last):
...
AttributeError: 'Dict' object has no attribute 'empty'
'''
def __init__(self, **kw):
super(Dict, self).__init__(**kw)
def __getattr__(self, key):
try:
return self[key]
except KeyError:
raise AttributeError(r"'Dict' object has no attribute '%s'" % key)
def __setattr__(self, key, value):
self[key] = value
if __name__=='__main__':
import doctest
doctest.testmod()
Run python mydict2.py:
$ python mydict2.py
There is no output. This shows that the doctest we wrote runs correctly. If there is a problem with the program, such as __getattr__()commenting out the method and running it again, an error will be reported:
$ python mydict2.py
**********************************************************************
File "/Users/michael/Github/learn-python3/samples/debug/mydict2.py", line 10, in __main__.Dict
Failed example:
d1.x
Exception raised:
Traceback (most recent call last):
...
AttributeError: 'Dict' object has no attribute 'x'
**********************************************************************
File "/Users/michael/Github/learn-python3/samples/debug/mydict2.py", line 16, in __main__.Dict
Failed example:
d2.c
Exception raised:
Traceback (most recent call last):
...
AttributeError: 'Dict' object has no attribute 'c'
**********************************************************************
1 items had failures:
2 of 9 in __main__.Dict
***Test Failed*** 2 failures.
Notice the last 3 lines of code. When the module is imported normally, doctest is not executed. Doctest is only executed when run directly from the command line. Therefore, there is no need to worry about doctest being executed in a non-test environment.
practise
Write a doctest for the function fact(n)and execute it:
def fact(n):
'''
Calculate 1*2*...*n
>>> fact(1)
1
>>> fact(10)
3628800
>>> fact(-1)
Traceback (most recent call last):
...
ValueError
'''
if n < 1:
raise ValueError()
if n == 1:
return 1
return n * fact(n - 1)
if __name__ == '__main__':
import doctest
doctest.testmod()
summary
Doctest is very useful. It can not only be used for testing, but also can be used directly as sample code. Through some document generation tools, comments containing doctest can be automatically extracted. When the user reads the document, he also sees the doctest.
Reference link: Errors, debugging and testing - Liao Xuefeng's official website (liaoxuefeng.com)