foreword
When it comes to Python's unit testing framework, the first thing that comes to mind for friends who have been in contact with Python is unittest. Indeed, as Python's standard library, it's excellent and widely used in various projects. but you know what? In fact, among many Python projects, the mainstream unit testing framework is far more than this one.
This series of articles will introduce you to the currently popular Python unit testing frameworks, talk about their functions and characteristics, and compare their similarities and differences, so that you can weigh the pros and cons and choose the best one when facing different scenarios and different needs. Unit testing framework.
This article takes Python 3 as an example by default. If some features are not available or different in Python 2, they will be specially explained.
1. Introduction
The unittest unit testing framework was first inspired by JUnit, and has a similar style to mainstream unit testing frameworks in other languages.
It supports test automation, multiple test cases share the front (setUp) and cleanup (tearDown) code, aggregate multiple test cases into a test set, and separate the testing and reporting frameworks.
2. Use case writing
The following simple example comes from the official documentation to test three string methods: upper, isupper, and split:
import unittest
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
if __name__ == '__main__':
unittest.main()
In the above example, a test case is created by inheriting unittest.TestCase. In this class, define the method starting with test, and the test framework will execute it as an independent test.
Each use case uses the built-in assertion method of unittest to determine whether the behavior of the object under test is as expected, such as:
In the test_upper test, use assertEqual to check if it is the expected value
In the test_isupper test, use assertTrue or assertFalse to verify that the condition is met
In the test_split test, use assertRaises to verify that a specific exception is thrown
Some people may be curious, why not use the built-in assertion statement assert, but provide so many assertion methods and use it? The reason is that by using the assertion method provided by unittest, the test framework can aggregate all test results and generate an informative test report after the run. Although the direct use of assert can also achieve the purpose of verifying whether the object under test meets expectations, but when the use case fails, the error information is not rich enough.
3. Use case discovery and execution
unittest supports automatic (recursive) discovery of use cases:
By default, all test cases conforming to test*.py are found in the current directory
Use python -m unittest or python -m unittest discover
Specify the directory to be automatically discovered by the -s parameter, and the -p parameter specifies the name pattern of the use case file
python -m unittest discover -s project_directory -p "test_*.py"
Specify the name pattern of auto-discovered directories and use case files via positional parameters
python -m unittest discover project_directory "test_*.py"
unittest supports the execution of specified use cases:
Specify the test module
python -m unittest test_module1 test_module2
Specify test class
python -m unittest test_module.TestClass
specify test method
python -m unittest test_module.TestClass.test_method
Specify test file path (Python 3 only)
python -m unittest tests/test_something.py
Fourth, the test fixture (Fixtures)
The test fixture is the test pre-(setUp) and clean-up (tearDown) methods.
The pre-test method setUp() is used to do some preparatory work, such as establishing a database connection. It will be automatically called by the test framework before the test case is executed.
The test cleanup method tearDown() is used to do some cleanup work, such as disconnecting the database connection. It will be called automatically by the test framework after the execution of the test case is completed (including failure cases).
Test prep and cleanup methods can have different execution levels.
1 Effective level: test method
If we want to execute the test pre-test and clean-up methods before and after each test method, then we need to define setUp() and tearDown() in the test class:
class MyTestCase(unittest.TestCase):
def setUp(self):
pass
def tearDown(self):
pass2 Effective level: test class
If we want to execute the pre-method only once in a single test class, then execute all the tests in the test class, and finally execute the cleanup method, then we need to define setUpClass() and tearDownClass () in the test class :
class MyTestCase(unittest.TestCase):
def setUpClass(self):
pass
def tearDownClass(self):
pass
3 Validity level: test module
If we want to execute the pre-method only once in a single test module, then execute all tests of all test classes in the module, and finally execute the cleanup method, then we need to define setUpModule() and tearDownModule() in the test module :
def setUpModule():
pass
def tearDownModule():
pass5. Skipping tests and predicting failures
unittest supports skipping tests directly or by condition, and also supports predicting test failures:
Skip tests directly via skip decorator or SkipTest
Conditionally skip or not skip tests via skipIf or skipUnless
Expected test failure via expectedFailure
class MyTestCase(unittest.TestCase):
@unittest.skip("直接跳过")
def test_nothing(self):
self.fail("shouldn't happen")
@unittest.skipIf(mylib.__version__ < (1, 3),
"满足条件跳过")
def test_format(self):
# Tests that work for only a certain version of the library.
pass
@unittest.skipUnless(sys.platform.startswith("win"), "满足条件不跳过")
def test_windows_support(self):
# windows specific testing code
pass
def test_maybe_skipped(self):
if not external_resource_available():
self.skipTest("跳过")
# test code that depends on the external resource
pass
@unittest.expectedFailure
def test_fail(self):
self.assertEqual(1, 0, "这个目前是失败的")Six, sub-test
Sometimes, you may want to write such a test: passing in different parameters in a test method to test the same piece of logic, but it will be regarded as a test, but if subtests are used, it can be regarded as N ( is the number of parameters) tests. Here is an example
class NumbersTest(unittest.TestCase):
def test_even(self):
"""
Test that numbers between 0 and 5 are all even.
"""
for i in range(0, 6):
with self.subTest(i=i):
self.assertEqual(i % 2, 0)The method used in the example with self.subTest(i=i) defines subtests. In this case, even if a single subtest fails to execute, it will not affect the execution of subsequent subtests. This way, we can see that three subtests fail in the output:
======================================================================
FAIL: test_even (__main__.NumbersTest) (i=1)
----------------------------------------------------------------------
Traceback (most recent call last):
File "subtests.py", line 32, in test_even
self.assertEqual(i % 2, 0)
AssertionError: 1 != 0
======================================================================
FAIL: test_even (__main__.NumbersTest) (i=3)
----------------------------------------------------------------------
Traceback (most recent call last):
File "subtests.py", line 32, in test_even
self.assertEqual(i % 2, 0)
AssertionError: 1 != 0
======================================================================
FAIL: test_even (__main__.NumbersTest) (i=5)
----------------------------------------------------------------------
Traceback (most recent call last):
File "subtests.py", line 32, in test_even
self.assertEqual(i % 2, 0)
AssertionError: 1 != 07. Test result output
Based on the example mentioned in the simple example section, to illustrate unittest the result output after running the test.
The output by default is very simple, showing how many cases were run and how long it took:
...
----------------------------------------------------------------------
Ran 3 tests in 0.000s
OKBy specifying -v parameters, detailed output can be obtained. In addition to the default output content, the use case name is additionally displayed:
test_isupper (__main__.TestStringMethods) ... ok
test_split (__main__.TestStringMethods) ... ok
test_upper (__main__.TestStringMethods) ... ok
----------------------------------------------------------------------
Ran 3 tests in 0.001s
OKAssuming test_upper the test fails, in verbose output mode, the result is as follows:
test_isupper (tests.test.TestStringMethods) ... ok
test_split (tests.test.TestStringMethods) ... ok
test_upper (tests.test.TestStringMethods) ... FAIL
======================================================================
FAIL: test_upper (tests.test.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Uvsers/prodesire/projects/tests/test.py", line 6, in test_upper
self.assertEqual('foo'.upper(), 'FOO1')
AssertionError: 'FOO' != 'FOO1'
- FOO
+ FOO1
? +
----------------------------------------------------------------------
Ran 3 tests in 0.001s
FAILED (failures=1)If we change test_upper the test method to , the output of the test result will lack contextual information that is helpful for troubleshooting:self.assertEqualassert
test_isupper (tests.test.TestStringMethods) ... ok
test_split (tests.test.TestStringMethods) ... ok
test_upper (tests.test.TestStringMethods) ... FAIL
======================================================================
FAIL: test_upper (tests.test.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/prodesire/projects/tests/test.py", line 6, in test_upper
assert 'foo'.upper() == 'FOO1'
AssertionError
----------------------------------------------------------------------
Ran 3 tests in 0.001s
FAILED (failures=1)If you want to generate a report in HTML format, you need to use a third-party library (such as HtmlTestRunner) to operate.
After installing the third-party library, you cannot directly use python -m unittest plus a method similar to --html report.html to generate HTML reports, but you need to write a small amount of code yourself to run test cases and then get HTML reports. For details, please refer to the HtmlTestRunner instruction manual.
Eight, the difference between unittest and pytest
1 Use case writing rules
unittest provides test cases, test suites, test fixtures, and test runner-related classes to make testing more clear, convenient, and controllable. When using unittest to write use cases, the following rules must be followed:
(1) The test file must first import unittest
(2) The test class must inherit unittest.TestCase
(3) The test method must start with "test_"
(4) The test class must have the unittest.main() method
pytest is a third-party testing framework for python and an extended framework based on unittest, which is more concise and efficient than unittest. To write use cases using pytest, the following rules must be followed:
(1) The test file name must start with "test_" or end with "_test" (eg: test_ab.py)
(2) The test method must start with "test_".
(3) The test class name starts with "Test".
Summary: pytest can execute unittest-style test cases without modifying any code of unittest use cases, and has better compatibility. The pytest plug-in is rich, such as the flask plug-in, which can be used to re-run the use case when an error occurs; there is also an xdist plug-in, which can be used for parallel execution of the device.
2 Use case pre and post
(1) unittest provides setUp/tearDown, which runs once before and after each use case runs. setUpClass and tearDownClass are run only once before and after the use case is executed.
# unittset前置条件.py
import unittest
class Test(unittest.TestCase): # 继承unittest中的TestCase
@classmethod
def setUpClass(cls) -> None: # setUpClass：所有用例执行之前会执行一次，如：打开文件，链接数据库
print('setUpClass')
@classmethod
def tearDownClass(cls) -> None: # tearDownClass：所有用例执行之后会执行一次，如：注册后删掉数据
print('tearDownClass')
@classmethod
def setUp(self) -> None: # setUp：每条用例执行前都会先执行setUp，如：
print('setUp')
@classmethod
def tearDown(self) -> None: # tearDown：每条用例执行后都会先执行tearDown，如：
print('tearDown')
def testB(self): # 用例执行顺序：以test后字母开头排序
print('testB')
def testA(self):
print('testA')
def testZ(self):
print('testZ')
if __name__ == "__main__":
# unittest.main() # 不产生测试报告
passIts execution results are as follows:
Ran 3 tests in 0.003s
Launching unittests with arguments python -m unittest 用例前置条件.Test in /Users/ray/PycharmProjects/day10
OK
setUpClass
tearDownClass
Process finished with exit code 0
setUp
testA
tearDown
setUp
testB
tearDown
setUp
testZ
tearDownIts execution results are as follows:
collected 4 items
test_module.py setup_module：整个.py模块只执行一次
setup_function：每个用例开始前都会执行
正在执行测试模块----test_one
.teardown_function：每个用例结束后都会执行
setup_function：每个用例开始前都会执行
正在执行测试模块----test_two
Fteardown_function：每个用例结束后都会执行
setup_class：所有用例执行之前
setup：每个用例开始前都会执行
正在执行测试类----test_three
.teardown：每个用例结束后都会执行
setup：每个用例开始前都会执行
正在执行测试类----test_four
Fteardown：每个用例结束后都会执行
teardown_class：所有用例执行之后
teardown_module：整个test_module.py模块只执行一次Method 2: pytest's fixture method
# conftest.py
# -*- coding: utf-8 -*-
import pytest
@pytest.fixture(scope="function")
def login():
print("请先输入账号和密码，然后登陆")
yield
print("退出登陆")# test_1.py
# -*- coding: utf-8 -*-
import pytest
def test_fix1(login):
print("test_fix1 in test_1.py：需要登陆再执行操作")
def test_fix2():
print("test_fix2 in test_1.py：不需要登陆再执行操作")
def test_fix3(login):
print("test_fix3 in test_1.py：需要登陆再执行操作")
if __name__ == "__main__":
pytest.main(['-s', 'test_1.py'])# test_2.py
# -*- coding: utf-8 -*-
import pytest
def test_fix3():
print("test_fix3 in test_2.py：不需要登陆再执行操作")
def test_fix4(login):
print("test_fix4 in test_2.py：需要登陆再执行操作")
if __name__ == "__main__":
pytest.main(['-s', 'test_2.py'])Its execution results are as follows:
pytest -s test_1.py
collected 3 items
test_1.py 请先输入账号和密码，然后登陆
test_fix1 in test_1.py：需要登陆再执行操作
.退出登陆
test_fix2 in test_1.py：不需要登陆再执行操作
.请先输入账号和密码，然后登陆
test_fix3 in test_1.py：需要登陆再执行操作
.退出登陆3 Affirmations
(1) unittest provides assertEqual, assertIn, assertTrue, assertFalse.
assertEqual: Determine whether the first parameter and the second parameter of the assertion are equal, and if they are not equal, the test fails
Usage: assertIn(key, container, message)
key: A string to check for existence in the given container
container: the string in which to search for the key string
message: A string statement that is the message to display when the test message fails.
assertIn: Used in unit tests to check if a string is contained within another string. This function will take three string parameters as input and return a boolean value based on the assertion condition. It will return true if key is contained in the container string, false otherwise.
Usage: assertIn(key, container, message)
Parameters: assertIn() accepts the description of the following three parameters:
key: A string to check for existence in the given container
container: the string in which to search for the key string
message: A string statement that is the message to display when the test message fails.
assertTrue: judge whether it is true
assertFalse: judge whether it is false
(1) pytest directly uses the assert expression.
assert: Used to judge an expression and trigger an exception when the expression condition is false.
4 report
(1) unittest uses the HTMLTestRunnerNew library.
(2) pytest has pytest-HTML and allure plugins.
5 fail rerun
(1) unittest does not have this function.
(2) pytest supports rerun of use case execution failure, pytest-rerunfailures plugin.
6 Parameterization
(1) unittest needs to rely on ddt library or parameterized library.
# 单元测试.py
import unittest
import myFunction
import HTMLTestRunner
import HTMLTestRunnerNew # 测试报告丰富版本
import parameterized # 参数化
class TestAdd(unittest.TestCase):
'''测试add方法'''
@parameterized.parameterized.expand( # 传参为二维数组
[[1, 2, 3, '参数化1'],
[-1, 2, 3, '参数化2'],
[2, 4, 7, '参数化3']]
)
def testParamAdd(self, a, b, c, desc):
self._testMethodDoc = desc # 使用这个_testMethodDoc参数传递
result = myFunction.add(a, b)
self.assertEqual(c, result, '预期结果是%s,实际结果是%s' % (c, result))(2) pytest directly uses the @pytest.mark.parametrize decorator.
@allure.epic("SOS接口自动化测试")
class TestCaseRunner:
@allure.feature("过程管理/风险处理/干预任务报表(新)-查询")
@pytest.mark.parametrize("case", CaseExcuteUtils.get_case_list("soscases/progressManagement/taskreport", case_tag))
def test_task_report(self, case):
"""
参数化执行测试用例
:param case:
:return:
"""
print(case.description)
allure.dynamic.title(case.description)
CaseExcuteUtils.excute_case(case, data)7 Use case classification execution
(1) unittest executes all test cases by default, and can execute some module test cases by loading testsuite;
(2) pytest can mark test cases through @pytest.mark, execute the command and add the parameter "-m" to run the marked use case.
Nine. Summary
unittest, as a unit testing framework provided by the Python standard library, is easy to use and powerful, and can satisfy daily testing needs. It is the best choice for unit testing without introducing third-party libraries.
In the next article, we will introduce the third-party unit testing framework nose and nose2, and talk about its improvements compared to unittest, so that many developers choose it first
Finally, I would like to thank everyone who has read my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, you can take it away if you need it:
These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey, and I hope it can help you! Partners can click the small card below to receive