目录
使用和调用
通过python -m pytest调用pytest
你可以通过python的解释器来执行测试:
python -m pytest [...]
但是,这和直接执行pytest [...]
命令的效果几乎是一模一样的;
pytest执行结束时返回的状态码
pytest
命令执行结束,可能会返回以下六种状态码:
- 0(OK):所有收集到的用例测试通过
- 1(TESTS_FAILED):有用例测试失败
- 2(INTERRUPTED):用户打断测试执行
- 3(INTERNAL_ERROR):测试执行的过程中,发生内部错误
- 4(USAGE_ERROR):
pytest
命令使用错误 - 5(NO_TESTS_COLLECTED):没有收集到测试用例
它们在枚举类_pytest.main.ExitCode中声明。并且,其作为公开API的一部分,能够直接引入和访问。
from pytest import ExitCode
获取帮助信息
pytest --version # 查看版本号和pytest的引入路径
pytest -h # 查看帮助信息
最多允许失败的测试用例数
当达到最大上限时,退出执行;如未配置,则没有上限;
pytest -x # 遇到第一个失败时,退出执行
pytest --maxfail==2 # 遇到第二个失败时,退出执行
执行指定的测试用例
pytest支持多种方式来执行特定的测试用例:
执行指定模块中的测试用例:
pytest test_mod.py
执行指定目录下的测试用例:
pytest testing/
执行文件名、类名或者函数名中包含特定关键字的测试用例:
yaomengdeMacBook-Air:src yaomeng$ pytest -k "_class and not two"
============================================================ test session starts =============================================================
platform darwin -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: /Users/yaomeng/Private/Projects/pytest-chinese-doc/src
plugins: repeat-0.8.0
collected 5 items / 4 deselected / 1 selected
test_class.py . [100%]
====================================================== 1 passed, 4 deselected in 0.03s =======================================================
执行src/
目录下,名字包含_class
但不包含two
的测试用例,即执行test_class.py
模块中的test_one
用例。
注意:python的关键字不可以应用在
-k
选项中,例如,class
、def
等。
执行指定nodeid的测试用例:
pytest为每一个收集到的测试用例指定一个唯一的nodeid
。其由模块名加说明符构成,中间以::
间隔。其中,说明符可以是类名、函数名以及由parametrize
标记赋予的参数;
# test_nodeid.py
import pytest
def test_one():
print('test_one')
assert 1
class TestNodeId:
def test_one(self):
print('TestNodeId::test_one')
assert 1
@pytest.mark.parametrize('x,y', [(1, 1), (3, 4)])
def test_two(self, x, y):
print(f'TestNodeId::test_two::{x} == {y}')
assert x == y
执行test_one
用例:
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest -s test_nodeid.py::test_one
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src
collected 1 item
test_nodeid.py test_one
.
================================================== 1 passed in 0.02s ==================================================
执行TestNodeId
中的test_one
用例:
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest -s test_nodeid.py::TestNodeId::test_one
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src
collected 1 item
test_nodeid.py TestNodeId::test_one
.
================================================== 1 passed in 0.02s ==================================================
执行TestNodeId
中的test_two
用例,并指定参数x
、y
的值:
λ pytest -s test_nodeid.py::TestNodeId::test_two[3-4]
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src
collected 1 item
test_nodeid.py TestNodeId::test_two::3 == 4
F
====================================================== FAILURES ======================================================= ______________________________________________ TestNodeId.test_two[3-4] _______________________________________________
self = <test_nodeid.TestNodeId object at 0x00000152613C49B0>, x = 3, y = 4
@pytest.mark.parametrize('x,y', [(1, 1), (3, 4)])
def test_two(self, x, y):
print(f'TestNodeId::test_two::{x} == {y}')
> assert x == y
E assert 3 == 4
test_nodeid.py:37: AssertionError
================================================== 1 failed in 0.05s ==================================================
这里对参数
x
、y
赋值的形式是[3-4]
,中间以-间隔;单个或多个参数的赋值形式以此类比;并且,只能为[1-1]
或者[3-4]
,其它的会报错;
执行指定标记的用例
pytest -m slow
执行所有用@pytest.mark.slow
装饰器标记的测试用例。
执行指定包中的测试用例
/d/Personal Files/Python/pytest-chinese-doc (5.1.2)
λ pytest --pyargs src
执行src包中所有的测试用例。
注意:src文件夹中要有一个__init__.py文件。
修改回溯信息的输出模式
pytest回溯信息的输出一共有六种模式:auto/long/short/line/native/no,用--tb
选项指定;
pytest -l, --showlocals # 打印本地变量
pytest --tb=auto # 默认模式
pytest --tb=long # 尽可能详细的输出
pytest --tb=short # 更简短的输出
pytest --tb=line # 每个失败信息总结在一行中
pytest --tb=native # python的标准输出
pytest --tb=no # 不打印失败信息
--full-trace
是一种比--tb=long
更详细的输出模式。它甚至能观察到用户打断执行(Ctrl+C
)时的回溯信息,而上述六种模式默认是不输出信息的。
总结报告
-r
选项可以在执行结束后,打印一个简短的总结报告。在执行的测试用例很多时,可以让你对结果有个清晰的了解:
# test_report.py
import pytest
@pytest.fixture
def error_fixture():
assert 0
def test_ok():
print("ok")
def test_fail():
assert 0
def test_error(error_fixture):
pass
def test_skip():
pytest.skip("skipping this test")
def test_xfail():
pytest.xfail("xfailing this test")
@pytest.mark.xfail(reason="always xfail")
def test_xpass():
pass
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest -ra test_report.py
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src
collected 6 items
test_report.py .FEsxX [100%]
======================================================= ERRORS ======================================================== ____________________________________________ ERROR at setup of test_error _____________________________________________
@pytest.fixture
def error_fixture():
> assert 0
E assert 0
test_report.py:27: AssertionError
====================================================== FAILURES ======================================================= ______________________________________________________ test_fail ______________________________________________________
def test_fail():
> assert 0
E assert 0
test_report.py:35: AssertionError
=============================================== short test summary info ===============================================
SKIPPED [1] D:\Personal Files\Python\pytest-chinese-doc\src\test_report.py:44: skipping this test
XFAIL test_report.py::test_xfail
reason: xfailing this test
XPASS test_report.py::test_xpass always xfail
ERROR test_report.py::test_error - assert 0
FAILED test_report.py::test_fail - assert 0
======================== 1 failed, 1 passed, 1 skipped, 1 xfailed, 1 xpassed, 1 error in 0.10s ========================
-r
选项后面要紧接这一个参数,用于过滤显示测试用例的结果。a
表示除了成功的其它所有的测试用例。
以下是所有有效的字符参数:
- f:失败的
- E:出错的
- s:跳过执行的
- x:跳过执行,并标记为xfailed的
- X:跳过执行,并标记为xpassed的
- p:测试通过的
- P:测试通过,并且有输出信息的;即用例中有
print
等 - a:除了测试通过的,其他所有的;即除了
p
和P
的 - A:所有的
上述字符参数可以叠加使用,例如:我们期望过滤出失败的和未执行的:
pytest -rfs
失败时加载PDB(Python Debugger)环境
PDB是python内建的诊断器,pytest允许通过以下命令在执行失败时进入这个诊断器模式:
pytest --pdb
pytest会在测试用例失败(或者Ctrl+C)时,调用这个诊断器:
luyao@NJ-LUYAO-T460 /d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest --pdb test_class.py
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src
collected 2 items
test_class.py .F
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
self = <src.test_class.TestClass object at 0x00000269030ECB70>
def test_two(self):
x = 'hello'
> assert hasattr(x, 'check')
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_class.py:30: AssertionError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> d:\personal files\python\pytest-chinese-doc\src\test_class.py(30)test_two()
-> assert hasattr(x, 'check')
(Pdb) x
'hello'
(Pdb)
此时,你可以访问测试用例的本地变量
x
;
并且,失败的信息存储在sys.last_value, sys.last_type, sys.last_traceback
变量中,你可以在交互环境中访问它们:
(Pdb) import sys
(Pdb) sys.last_value
AssertionError("assert False\n + where False = hasattr('hello', 'check')")
(Pdb) sys.last_type
<class 'AssertionError'>
(Pdb) sys.last_traceback
<traceback object at 0x00000269030F0908>
(Pdb)
开始执行时就加载PDB环境
通过以下命令,pytest允许你在每个测试用例执行时,就加载PDB环境:
pytest --trace
设置断点
在测试用例代码中添加import pdb;pdb.set_trace()
,当其被调用时,pytest会停止这条用例的输出:
- 其他用例不受影响;
- 通过
continue
命令,退出PDB环境,并继续执行用例;
使用内置的中断函数
python 3.7介绍了一个内置breakpoint()
函数。pytest可以在以下场景中支持使用:
- 当
breakpoint()
被调用,并且PYTHONBREAKPOINT
未设置时,pytest会使用内部自定义的PDB代替系统的PDB; - 测试执行结束时,自动切回系统自带的PDB;
- 当加上
--pdb
选项时,breakpoint()
和测试发生错误时,都会调用内部自定义的PDB; --pdbcls
选项允许指定一个用户自定义的PDB类;
分析测试执行时间
获取执行最慢的10个测试用例:
pytest --durations=10
默认情况下,pytest不会显示执行时间<0.01s的测试用例,可以使用-vv
选项查看它们;
错误句柄
5.0版本新增特性
在测试执行中发生段错误或者超时的情况下,faulthandler
标准模块可以转储python的回溯信息;
它在pytest的执行中默认使能,使用-p no:faulthandler
选项可以关闭它;
同样,faulthandler_timeout=X
配置项,可用于当测试用例的完成时间超过X
秒时,转储所有线程的python回溯信息;
# pytest.ini
[pytest]
faulthandler_timeout=5
配置测试执行的超时时间是5秒;
# test_fault_handler.py
import time
def test_fault_handler():
time.sleep(10)
assert 1
测试用例中添加等待10秒的操作;
yaomengdeMacBook-Air:src yaomeng$ pytest test_fault_handler.py
============================================================ test session starts =============================================================
platform darwin -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: /Users/yaomeng/Private/Projects/pytest-chinese-doc/src, inifile: pytest.ini
plugins: repeat-0.8.0
collected 1 item
test_fault_handler.py Timeout (0:00:05)!
Thread 0x000000010d5895c0 (most recent call first):
File "/Users/yaomeng/Private/Projects/pytest-chinese-doc/src/test_fault_handler.py", line 26 in test_fault_handler
File "/usr/local/lib/python3.7/site-packages/_pytest/python.py", line 170 in pytest_pyfunc_call
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 81 in <lambda>
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 87 in _hookexec
File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 289 in __call__
File "/usr/local/lib/python3.7/site-packages/_pytest/python.py", line 1423 in runtest
File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 117 in pytest_runtest_call
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 81 in <lambda>
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 87 in _hookexec
File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 289 in __call__
File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 192 in <lambda>
File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 220 in from_call
File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 192 in call_runtest_hook
File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 167 in call_and_report
File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 87 in runtestprotocol
File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 72 in pytest_runtest_protocol
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 81 in <lambda>
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 87 in _hookexec
File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 289 in __call__
File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 256 in pytest_runtestloop
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 81 in <lambda>
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 87 in _hookexec
File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 289 in __call__
File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 235 in _main
File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 191 in wrap_session
File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 228 in pytest_cmdline_main
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 81 in <lambda>
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 87 in _hookexec
File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 289 in __call__
File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 78 in main
File "/usr/local/bin/pytest", line 10 in <module>
. [100%]
============================================================= 1 passed in 10.02s =============================================================
在执行刚超过5秒的时候会打印出回溯信息。但不会中断测试的执行;
yaomengdeMacBook-Air:src yaomeng$ pytest -p no:faulthandler test_fault_handler.py
============================================================ test session starts =============================================================
platform darwin -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: /Users/yaomeng/Private/Projects/pytest-chinese-doc/src, inifile: pytest.ini
plugins: repeat-0.8.0
collected 1 item
test_fault_handler.py . [100%]
============================================================= 1 passed in 10.02s =============================================================
当去使能faulthandler
的时,超时并不会触发回溯信息的打印;
注意:这个功能是从pytest-faulthandler插件合并而来的,但是有两点不同:
- 去使能时,使用
-p no:faulthandler
代替原来的--no-faulthandler
;- 使用
faulthandler_timeout
配置项代替--faulthandler-timeout
命令行选项来配置超时时间。当然,你也可以使用-o faulthandler_timeout=X
在命令行配置;
创建JUnitXML格式的结果报告
使用如下命令,可以创建一个能被Jenkins或者其他CI工具读取的测试结果文件:
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest test_class.py --junitxml=../report/test_class.xml
它会在指定目录下创建一个XML文件:
<!--test_class.xml-->
<?xml version="1.0" encoding="utf-8"?>
<testsuites>
<testsuite errors="0" failures="1" hostname="NJ-LUYAO-T460" name="pytest" skipped="0" tests="2" time="0.058"
timestamp="2019-09-10T09:53:07.554001">
<testcase classname="test_class.TestClass" file="test_class.py" line="23" name="test_one" time="0.001">
</testcase>
<testcase classname="test_class.TestClass" file="test_class.py" line="27" name="test_two" time="0.002">
<failure message="AssertionError: assert False
+ where False = hasattr('hello', 'check')">self = <src.test_class.TestClass object at
0x000001773E688C50>
def test_two(self):
x = 'hello'
> assert hasattr(x, 'check')
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_class.py:30: AssertionError</failure>
</testcase>
</testsuite>
</testsuites>
你可以在pytest.ini
文件中设置junit_suite_name
,来修改xml文件中testsuite
根节点的name
信息:
# pytest.ini
[pytest]
junit_suite_name = pytest_chinese_doc
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
pytest test_class.py test_sample.py --junitxml=../report/test_mysuitename.xml
<!--test_mysuitename.xml-->
<?xml version="1.0" encoding="utf-8"?>
<testsuites>
<testsuite errors="0" failures="2" hostname="NJ-LUYAO-T460" name="pytest_chinese_doc" skipped="0" tests="3"
time="0.070" timestamp="2019-09-10T10:07:29.078541">
<testcase classname="test_class.TestClass" file="test_class.py" line="23" name="test_one" time="0.000">
</testcase>
<testcase classname="test_class.TestClass" file="test_class.py" line="27" name="test_two" time="0.001">
<failure message="AssertionError: assert False
+ where False = hasattr('hello', 'check')">self = <src.test_class.TestClass object at
0x000001C7F1155550>
def test_two(self):
x = 'hello'
> assert hasattr(x, 'check')
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_class.py:30: AssertionError</failure>
</testcase>
<testcase classname="test_sample" file="test_sample.py" line="26" name="test_sample" time="0.001">
<failure message="assert 4 == 5
+ where 4 = func(3)">def test_sample():
> assert func(3) == 5
E assert 4 == 5
E + where 4 = func(3)
test_sample.py:28: AssertionError</failure>
</testcase>
</testsuite>
</testsuites>
可以看到,<testsuite errors="0" failures="2" hostname="NJ-LUYAO-T460" name="pytest_chinese_doc"
的name信息已经从默认的pytest变成我们期望的pytest_chinese_doc;
注意:这是4.0版本的新增配置项;
JUnit XML规定time
属性应该表明测试用例执行的全部耗时,包含setup
和teardown
中的操作,这也是pytest的默认行为;如果你只想记录测试用例执行的时间,只需要做如下配置:
# pytest.ini
junit_duration_report = call
record_property
如果你想在一个测试用例的报告中添加一些额外的属性节点,可以使用record_property
fixture:
# test_record_property.py
def test_record_property(record_property):
record_property('example_key', 1)
assert 1
我们期望添加一个example_key
属性节点;
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest test_record_property.py --junitxml=../report/test_record_property.xml
<!--test_record_property.xml-->
<?xml version="1.0" encoding="utf-8"?>
<testsuites>
<testsuite errors="0" failures="0" hostname="NJ-LUYAO-T460" name="pytest_chinese_doc" skipped="0" tests="1"
time="0.025" timestamp="2019-09-10T10:24:20.796619">
<testcase classname="test_record_property" file="test_record_property.py" line="22" name="test_record_property"
time="0.001">
<properties>
<property name="example_key" value="1" />
</properties>
</testcase>
</testsuite>
</testsuites>
这样,我们就为test_record_property用例添加了一个example_key="1"
的新属性;
或者,你可以将这种功能集成到一个自定义的标记中,例如:允许为测试用例添加一个测试ID:
# conftest.py
def pytest_collection_modifyitems(session, config, items):
for item in items:
for marker in item.iter_markers(name='test_id'):
test_id = marker.args[0]
item.user_properties.append(('test_id', test_id))
为以下用例标记一个测试ID:
# test_testid.py
import pytest
@pytest.mark.test_id(10010)
def test_record_property(record_property):
record_property('example_key', 1)
assert 1
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest test_testid.py --junitxml=../report/test_testid.xml
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src, inifile: pytest.ini
collected 1 item
test_testid.py . [100%]
================================================== warnings summary ===================================================
d:\program files\python\python37\lib\site-packages\_pytest\mark\structures.py:324
d:\program files\python\python37\lib\site-packages\_pytest\mark\structures.py:324: PytestUnknownMarkWarning: Unknown pytest.mark.test_id - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
PytestUnknownMarkWarning,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
----------- generated xml file: D:\Personal Files\Python\pytest-chinese-doc\report\test_record_property.xml ----------- ============================================ 1 passed, 1 warnings in 0.03s ============================================
这里有个告警,提示我们没有注册自定义的标记,但并不影响执行。我们只需要在pytest.ini
中注册一下就可以了:
# pytest.ini
[pytest]
markers =
test_id: 为测试用例添加ID
<!--test_testid.xml-->
<?xml version="1.0" encoding="utf-8"?>
<testsuites>
<testsuite errors="0" failures="0" hostname="NJ-LUYAO-T460" name="pytest_chinese_doc" skipped="0" tests="1"
time="0.025" timestamp="2019-09-10T10:46:50.694771">
<testcase classname="test_testid" file="test_testid.py" line="24" name="test_record_property" time="0.002">
<properties>
<property name="test_id" value="10010" />
<property name="example_key" value="1" />
</properties>
</testcase>
</testsuite>
</testsuites>
可以看到,我们又为test_record_property用例添加了一个test_id="10010"
的新属性;
注意:这样使用可能会使XML文件不符合最新的JUnitXML的模式检查规则,导致在某些CI工具可能会发生未知的错误;
record_xml_attribute
如果你想在testcase
节点中添加一个新的属性,可以使用record_xml_attribute
fixture。如果这个属性之前已经存在了,那么就会用新值覆盖掉原来的值:
# test_record_attribute.py
def test_record_property(record_xml_attribute):
record_xml_attribute('example_key', 1)
record_xml_attribute('classname', 'overwrite_classname')
assert 1
添加一个新的example_key
属性,并修改classname
属性;
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest test_record_attribute.py --junitxml=../report/test_record_attribute.xml
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src, inifile: pytest.ini
collected 1 item
test_record_attribute.py . [100%]
================================================== warnings summary =================================================== test_record_attribute.py::test_record_attribute
test_record_attribute.py:23: PytestExperimentalApiWarning: record_xml_attribute is an experimental feature
def test_record_attribute(record_xml_attribute):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
---------- generated xml file: D:\Personal Files\Python\pytest-chinese-doc\report\test_record_attribute.xml ----------- ============================================ 1 passed, 1 warnings in 0.03s ============================================
<!--test_record_attribute.xml-->
<?xml version="1.0" encoding="utf-8"?>
<testsuites>
<testsuite errors="0" failures="0" hostname="NJ-LUYAO-T460" name="pytest_chinese_doc" skipped="0" tests="1"
time="0.026" timestamp="2019-09-10T11:00:57.818851">
<testcase classname="overwrite_classname" example_key="1" file="test_record_attribute.py" line="22"
name="test_record_attribute" time="0.002"></testcase>
</testsuite>
</testsuites>
可以看到,<testcase classname="overwrite_classname" example_key="1"
节点的classname被替换,并且新增了example_key属性;
注意:
record_xml_attribute
目前是一个实验性的功能,未来可能被更强大的API所替代,但功能本身会被保留。
注意:这样使用可能会使XML文件不符合最新的JUnitXML的模式检查规则,导致在某些CI工具可能会发生未知的错误;
record_testsuite_property
4.5版本新增功能
如果你想在testsuite
级别上添加属性节点,并且希望其作用于所有的测试用例,那么你可以在session
作用域中使用record_testsuite_property
fixture:
# conftest.py
@pytest.fixture(scope="session", autouse=True)
def log_global_env_facts(record_testsuite_property):
record_testsuite_property("ARCH", "PPC")
record_testsuite_property("STORAGE_TYPE", "CEPH")
record_testsuite_property
接收两个参数name
和value
以构成<property>
标签,其中,name
必须为字符串,value
会转换为字符串并进行XML转义;
# test_record_testsuite_property.py
class TestMe:
def test_foo(self):
assert True
log_global_env_facts
为session级别的fixture,并且,autouse
为True,所以并不需要事先引入;
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest test_record_testsuite_property.py --junitxml=../report/test_record_testsuite_property.xml
<!--test_record_testsuite_property.xml-->
<?xml version="1.0" encoding="utf-8"?>
<testsuites>
<testsuite errors="0" failures="0" hostname="NJ-LUYAO-T460" name="pytest_chinese_doc" skipped="0" tests="1"
time="0.029" timestamp="2019-09-10T11:20:46.123978">
<properties>
<property name="ARCH" value="PPC" />
<property name="STORAGE_TYPE" value="CEPH" />
</properties>
<testcase classname="test_record_testsuite_property.TestMe" file="test_record_testsuite_property.py" line="23"
name="test_foo" time="0.001"></testcase>
</testsuite>
</testsuites>
可以看到,在testsuite
节点中,新增了一个properties
属性节点,包含所有新增的属性,而且,它和testcase
节点是兄弟节点;
注意:这样生成的XML文件是符合最新的
xunit
标准的,这点和record_property
、record_xml_attribute
正好相反;
为测试报告提供在线的pastebin展示
目前,只实现了在http://bpaste.net上的展示功能;
为每一个失败用例创建URL
luyao@NJ-LUYAO-T460 /d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest test_class.py --pastebin=failed
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0 -- d:\program files\python\python37\python.exe
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src, inifile: pytest.ini
collected 2 items
test_class.py .F [100%]
====================================================== FAILURES =======================================================
_________________________________________________ TestClass.test_two __________________________________________________
self = <src.test_class.TestClass object at 0x000001AB97F29208>
def test_two(self):
x = 'hello'
> assert hasattr(x, 'check')
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_class.py:30: AssertionError
======================================== Sending information to Paste Service =========================================
test_class.py:30: AssertionError --> https://bpaste.net/show/HGeB
============================================= 1 failed, 1 passed in 5.64s =============================================
在浏览器中访问https://bpaste.net/show/HGeB,就能查看到失败的信息了。
为所有的测试用例创建URL
luyao@NJ-LUYAO-T460 /d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ pytest test_class.py --pastebin=all
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0 -- d:\program files\python\python37\python.exe
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src, inifile: pytest.ini
collected 2 items
test_class.py .F [100%]
====================================================== FAILURES =======================================================
_________________________________________________ TestClass.test_two __________________________________________________
self = <src.test_class.TestClass object at 0x00000218030F91D0>
def test_two(self):
x = 'hello'
> assert hasattr(x, 'check')
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_class.py:30: AssertionError
============================================= 1 failed, 1 passed in 0.06s =============================================
======================================== Sending information to Paste Service =========================================
pastebin session-log: https://bpaste.net/show/XdVT
在浏览器中访问https://bpaste.net/show/XdVT,就能查看到所有用例的执行信息了。
尽量早的加载插件
你可以在命令行中使用-p
选项,来尽量早的加载某一个插件:
pytest -p mypluginmodule
-p
选项接收一个name
参数,这个参数可以为:
- 一个完整的本地插件引入,例如:
myproject.plugins
,其必须是可以import的。 - 一个公共插件的名称,这是其注册时在
setuptools
中赋予的名字,例如:pytest-cov
去使能插件
你可以在命令行中使用-p
结合no:
,来去使能一个插件的加载,例如:
pytest -p no:doctest
在python代码中调用pytest
你可以在代码中直接调用pytest:
pytest.main()
这和你在命令行中执行pytest是一样的,不过它不会触发SystemExit
,而是返回exitcode:
# invoke_with_main.py
import time
def test_one():
time.sleep(10)
assert True
if __name__ == '__main__':
import pytest
ret = pytest.main([__file__])
print(ret == pytest.ExitCode.INTERRUPTED)
用例中有等待10秒的操作,当我们在这期间,打断执行(Ctr+C),我们期望pytest.main()
返回的状态码是INTERRUPTED
,也就是2
;
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ python invoke_with_main.py
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src, inifile: pytest.ini
collected 1 item
invoke_with_main.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
d:\personal files\python\pytest-chinese-doc\src\invoke_with_main.py:26: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --full-trace)
================================================ no tests ran in 2.11s ================================================
True
可以看到,pytest.main()
确实返回了INTERRUPTED
的状态码;
你也可以传递选项和参数:
pytest.main(["-x", "mytestdir"])
同样,也可以为pytest.main()
指定一个自定义的插件:
# invoke_with_plugins.py
class MyPlugin:
def pytest_sessionfinish(self):
print('*** test run reporting finishing')
if __name__ == '__main__':
import pytest
pytest.main(['test_class.py'], plugins=[MyPlugin()])
执行这个文件,会发现插件正常使能,并且它的钩子方法也被调用:
/d/Personal Files/Python/pytest-chinese-doc/src (5.1.2)
λ python invoke_with_plugins.py
================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.12.0
rootdir: D:\Personal Files\Python\pytest-chinese-doc\src, inifile: pytest.ini
collected 2 items
test_class.py .F [100%]
*** test run reporting finishing
====================================================== FAILURES =======================================================
_________________________________________________ TestClass.test_two __________________________________________________
self = <src.test_class.TestClass object at 0x00000236949A20B8>
def test_two(self):
x = 'hello'
> assert hasattr(x, 'check')
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_class.py:30: AssertionError
============================================= 1 failed, 1 passed in 0.06s =============================================
注意:调用
pytest.main()
会引入你的测试文件以及其引用的所有模块。由于python引入机制的缓存特性,当这些文件发生变化时,后续再调用pytest.main()
(在同一个程序执行过程中)时,并不会响应这些文件的变化。基于这个原因,我们不推荐在同一个程序中多次调用pytest.main()
(例如:为了重新执行测试;如果你确实有这个需求,或许可以考虑pytest-repeat插件);