pytest is a popular Python testing framework that provides many command-line options that give users more control over the test execution process. When using pytest for testing, it is very important to be familiar with pytest's command line options, which will help reduce errors and improve testing efficiency. This article will detail pytest's command-line options and give examples.
1. Classification of pytest command line options
The command-line options for pytest can be divided into three categories:
- Environment configuration options: These options are used to set the environment configuration of pytest, such as setting the log level, overriding the configuration file, setting the test mode, etc.
- Test filtering options: These options are used to filter and select test cases, such as specifying a test directory, selecting a specific test module, running a specified test function, etc.
- Test execution options: These options are used to control the test execution process of pytest, such as retrying failed tests, generating test reports, executing tests in parallel, etc.
Below, we will introduce the specific usage of these command line options one by one.
2. Environment configuration options
--version: print the pytest version number.
-h, --help: Display pytest command line options and usage instructions.
--verbose, -v: Increase the verbosity of the test result output.
--quiet, -q: Reduce the verbosity of the test result output.
--traceconfig: Show detailed information while parsing and loading configuration files.
--pdb: When a test fails or an exception occurs, enter the Python debugger.
--pdbcls: Type of custom debugger.
--capture: Set the standard output stream capture method of pytest, there are three optional values: sys, fd, no.
--norecursedirs: Set which directories are not searched for test cases.
--rootdir: Set the root directory of pytest.
--maxfail=n: Set to stop test execution after the nth test failure.
--junit-xml=PATH: Output test results to a JUnit XML file.
3. Test filter options
path: Specify the test directory or file, which can be a relative path or an absolute path. For example:
pytest tests/.-m: Select specific marked test cases for execution. For example:
pytest -m "slow".-k: Select a test case containing a keyword for execution. For example:
pytest -k "add or subtract".-x: Stop test execution when a test case fails.
--pdb: When a test fails or an exception occurs, enter the Python debugger.
--lf: Only rerun the test cases that failed the last test.
--ff: Only re-run the test cases that failed the last test, and re-run these test cases after all tests are completed.
--sw: Rerun the last modified test module.
--last-failed-no-failures: Only rerun the last failed test case (if there are no failed test cases, the test will not be executed).
--collect-only: Only execute the collection phase of test cases, do not run the execution phase of test cases.
--pdbcls: Type of custom debugger.
--trace: Display pytest's internal trace information.
--count: Run the specified number of test cases. For example:
pytest -v --count=10.
4. Test Execution Options
-n: Run test cases in parallel, which can be followed by a number to specify the degree of concurrency. For example:
pytest -n 4.-x: Stop test execution when a test case fails.
--maxfail=n: Set to stop test execution after the nth test failure.
--last-failed: Only rerun the last failed test case.
--failed-first: Run the previously failed test cases first.
--reruns=n: Rerun tests n times in case of test case failure.
--pdb: When a test fails or an exception occurs, enter the Python debugger.
--pdbcls: Type of custom debugger.
--junit-xml=PATH: Output test results to a JUnit XML file.
--html=PATH: Output test results to an HTML file.
--tb=long/short/line/native/no: Set the format of the output error message.
--capture=no: Disable capturing standard output and standard error, output them directly to the terminal.
--capture=sys/stdout/stderr: Set the standard output stream capture method of pytest, there are three optional values: sys, fd, no.
--show-capture=all/failed/no: Controls whether to show the captured stdout stream.
--disable-warnings: Disable pytest warning messages.
The above is the introduction of pytest's command line options. By using these options, you can better control the test execution process of pytest, reduce errors and improve test efficiency. Next, I will give code examples and running effects of some command line options.
5. Code example and running effect
1.--version: print the pytest version number.
This option can be used by executing the pytest --version command, the code example is as follows:
$ pytest --version
This is pytest version 5.0.1, imported from /usr/local/lib/python3.6/site-packages/pytest.py
2.-h, --help: Display pytest command line options and usage instructions.
This option can be used by executing the pytest --help command, the code example is as follows:
$ pytest --help
usage: pytest [options] [file_or_dir] [file_or_dir] [...]positional arguments:
file_or_dir test file or root directory to search for testsoptional arguments:
-h, --help show this help message and exit
--version display pytest version and information about plugins
...
3.--verbose, -v: Increase the verbosity of the test result output.
This option can be used by executing the pytest -v command, the code example is as follows:
$ pytest -v
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract PASSED [100%]============================ 2 passed in 0.03s =============================
4. --quiet, -q: Reduce the verbosity of the test result output.
This option can be used by executing the pytest -q command, the code example is as follows:
$ pytest -q
..
2 passed in 0.03s
5. --traceconfig: Display detailed information when parsing and loading configuration files.
This option can be used by executing the pytest --traceconfig command, the code example is as follows:
$ pytest --traceconfig
using: pytest-5.0.1, py-1.8.0, pluggy-0.12.0
active plugins:
* pytest_cov-2.7.1 at /Users/user/anaconda3/envs/test/lib/python3.6/site-packages/pytest_cov/plugin.py
disabled plugins:
collector: <_pytest.main.Session object at 0x107cbe3c8>
6. --pdb: Enter the Python debugger when a test fails or an exception occurs.
This option can be used by executing the pytest --pdb command. When an error occurs in the test case, it will automatically enter the pdb debugger. The code example is as follows:
$ pytest --pdb
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]=================================== FAILURES ===================================
________________________________ test_subtract ________________________________def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)test_example.py:10: AssertionError
----------------------------- PDB -----------------------------
> /Users/user/example/test_example.py(10)test_subtract()
...
(Pdb)
7.--capture: Set the standard output stream capture method of pytest, there are three optional values: sys, fd, no.
This option can be used by executing the pytest --capture=sys command to redirect the standard output stream and standard error output to the log system of pytest. The code example is as follows:
$ pytest --capture=sys
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]=================================== FAILURES ===================================
________________________________ test_subtract ________________________________def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)test_example.py:10: AssertionError
[Capture] capturing logcall output to 'log': sys
8. --norecursedirs: Set which directories do not search for test cases.
This option can be used by executing the pytest --norecursedirs=examples command to exclude the examples directory from the path to search for test cases. The code example is as follows:
$ pytest --norecursedirs=examples
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract PASSED [100%]============================ 2 passed in 0.03s =============================
9. --rootdir: Set the root directory of pytest.
This option can be used by executing the pytest --rootdir=/path/to/project command, and set the root directory of pytest to /path/to/project. The code example is as follows:
$ pytest --rootdir=/path/to/project
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract PASSED [100%]============================ 2 passed in 0.03s =============================
10.--maxfail=n: Set to stop test execution after the nth test fails.
This option can be used by executing the pytest --maxfail=1 command. When the first test case fails, the test execution will be stopped. The code example is as follows:
$ pytest --maxfail=1
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]=================================== FAILURES ===================================
________________________________ test_subtract ________________________________def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)test_example.py:10: AssertionError
========================== 1 failed, 1 passed in 0.04s ==========================
11.--junit-xml=PATH: Output the test results to the JUnit XML file.
This option can be used by executing the pytest --junit-xml=test_results.xml command to output the test results to the test_results.xml file. The code example is as follows:
$ pytest --junit-xml=test_results.xml
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract PASSED [100%]============================ 2 passed in 0.03s =============================
12.path：指定测试目录或文件，可以是相对路径或绝对路径。例如：pytest tests/。
该选项可以通过执行 pytest tests/ 命令来使用，只运行 tests 目录下的测试用例，代码示例如下：
$ pytest tests/
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract PASSED [100%]============================ 2 passed in 0.03s =============================
13.-m：选择特定标记的测试用例进行执行。例如：pytest -m "slow"。
该选项可以通过在测试用例中使用 @pytest.mark 标记来进行使用，例如：
import pytest
@pytest.mark.slow
def test_slow():
pass
def test_not_slow():
pass
然后执行 pytest -m "slow" 命令来只运行标有 @pytest.mark.slow 标记的测试用例，代码示例如下：
$ pytest -m "slow"
test_example.py:3: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
@pytest.mark.slow
no tests ran in 0.00sPytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
14.-k: Select a test case containing a keyword for execution. For example: pytest -k "add or subtract".
This option can be used by executing the pytest -k "add or subtract" command, and only run the test cases whose test case name contains the string "add" or "subtract". The code example is as follows:
$ pytest -k "add or subtract"
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract PASSED [100%]============================ 2 passed in 0.03s =============================
15.-x: Stop test execution when a test case fails.
This option can be used by executing the pytest -x command. It will stop the test execution when the first test case fails. The code example is as follows:
$ pytest -x
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]=================================== FAILURES ===================================
________________________________ test_subtract ________________________________def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)test_example.py:10: AssertionError
========================== 1 failed, 1 passed in 0.04s ==========================
16.--lf: Only rerun the test cases that failed the last test.
This option can be used by executing the pytest --lf command. When an error occurs in the last test case, it will automatically rerun the failed test case. The code example is as follows:
$ pytest --lf
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]=================================== FAILURES ===================================
______________________ test_subtract[2-3] ______________________x = 2
y = 3def test_subtract(x, y):
assert subtract(x, y) == -1tests/test_example.py:10: AssertionError
------------------------------ rerun test call -------------------------------
test_example.py::test_subtract[x
17.--ff: Only rerun the test cases that failed the last test.
This option can be used by executing the pytest --ff command. When an error occurs in the last test case, it will automatically rerun the last failed test case. The code example is as follows:
$ pytest --ff
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract FAILED [100%]=================================== FAILURES ===================================
________________________________ test_subtract ________________________________def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)test_example.py:10: AssertionError
------------------------------- Captured stdout --------------------------------
subtraction of 2 and 3 is -1========================== 1 failed, 1 passed in 0.04s ==========================
18. --failed-first: Run the test case that failed the last test before all test cases.
This option can be used by executing the pytest --failed-first command. It will run the test cases that failed the last test first, and then run the remaining test cases. The code example is as follows:
$ pytest --failed-first
test_example.py::test_subtract FAILED [100%]
test_example.py::test_add PASSED=================================== FAILURES ===================================
________________________________ test_subtract ________________________________def test_subtract():
assert subtract(4, 3) == 1
> assert subtract(2, 3) == -1
E assert 2 == -1
E + where 2 = subtract(2, 3)test_example.py:10: AssertionError
----------------------------- Captured stdout -----------------------------
subtraction of 2 and 3 is -1========================== 1 failed, 1 passed in 0.03s ==========================
19.-n, --numprocesses: Specify the number of concurrent processes.
This option can be used by executing the pytest -n 4 command to assign test cases to 4 concurrent processes to run. The code example is as follows:
$ pytest -n 4
test_example.py::test_add PASSED [ 25%]
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract PASSED [ 75%]
test_example.py::test_subtract PASSED [100%]============================ 4 passed in 0.02s =============================
20. -d, --dist: enable distributed testing.
This option can be used by executing the pytest --dist=loadfile command, by loading multiple Python processes and distributing the test run among all processes, the code example is as follows:
$ pytest --dist=loadfile
test_example.py::test_add PASSED [ 25%]
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract PASSED [ 75%]
test_example.py::test_subtract PASSED [100%]============================ 4 passed in 0.02s =============================
21.--durations=n: Sort the test cases by running time and display the running time of the slowest n test cases.
This option can be used by executing the pytest --duration=2 command to display the running time of the two slowest test cases. The code example is as follows:
$ pytest --duration=2
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract PASSED [100%]======================== slowest 2 test durations =========================
0.01s call test_example.py::test_subtract
0.01s call test_example.py::test_add
============================ 2 passed in 0.02s =============================
22. --show-capture: Display the test case standard output stream and standard error output in the results.
This option can be used by executing the pytest --show-capture command to display the standard output stream and standard error output of the test case in the test results. The code example is as follows:
$ pytest --show-capture
test_example.py::test_add PASSED [ 50%]
test_example.py::test_subtract PASSED [100%]================================== Captured stdout =================================
subtraction of 4 and 3 is 1
subtraction of 2 and 3 is -1============================ 2 passed in 0.03s =============================
The above is the detailed introduction of pytest command line options and the corresponding code examples, hoping to help you better use pytest for testing.