Python is probably the hottest third party open source testing framework pytest

Author: HelloGitHub- Prodesire

HelloGitHub of "explain open source projects" series, Project address: https: //github.com/HelloGitHub-Team/Article

I. INTRODUCTION

This article is the third "talk Python unit testing framework", the first two were introduced standard library unittest and third-party unit test framework nose. As the last article in this series, the finale is the Python world's hottest third-party unit test framework: pytest.

pytest Project address: https: //github.com/pytest-dev/pytest

It has the following main features:

assert output detailed information assertion failure (no longer have to remember self.assert*the name of)
Auto discovery test modules and functions
Modular fixture for managing various types of test resources
Of unittestfully compatible for nose basic compatibility
Very rich plug-in system, there are more than 315 models third-party plug-ins , community prosperity

And described earlier unittestand nosethe same, we will introduce several aspects of the following pytestcharacteristics.

Second, prepared by Example

The same noseas pytestsupport functions, test class in the form of test cases. The biggest difference is that you can enjoy using the assertstatement asserted, did not worry it will noseor unittestlack of problems arising in the context of detailed information.

For example the following test example, so deliberately test_upperasserted not by:

import pytest

def test_upper():
    assert 'foo'.upper() == 'FOO1'

class TestClass:
    def test_one(self):
        x = "this"
        assert "h" in x

    def test_two(self):
        x = "hello"
        with pytest.raises(TypeError):
            x + []

Whereas when pytestthe time to execute the use cases, it outputs detailed (and a variety of colors) context information:

=================================== test session starts ===================================
platform darwin -- Python 3.7.1, pytest-4.0.1, py-1.7.0, pluggy-0.8.0
rootdir: /Users/prodesire/projects/tests, inifile:
plugins: cov-2.6.0
collected 3 items

test.py F..                                                                         [100%]

======================================== FAILURES =========================================
_______________________________________ test_upper ________________________________________

    def test_upper():
>       assert 'foo'.upper() == 'FOO1'
E       AssertionError: assert 'FOO' == 'FOO1'
E         - FOO
E         + FOO1
E         ?    +

test.py:4: AssertionError
=========================== 1 failed, 2 passed in 0.08 seconds ============================

Easy to see, pytestonly the output of the test code context information also measured output variable values. Compared to noseand unittest, pytestallow users an easier way to write test cases, but also get a richer and friendly test results.

Third, execution and use were found

unittestAnd nosesupported use cases discovered and implementation capacity, pytestare supported.
pytestAutomatic support (recursively) found with Example:

Default found that all comply with the current directory test_*.pyor *_test.pythe test case file to testtest the function at the beginning or at Testthe beginning of the test class with testtest methods at the beginning of
- Use pytestthe command
With nose2the same idea, by the configuration file to specify specific parameters, the model can be configured with the name of the file cases, classes and functions (fuzzy matching)

pytest Also support the implementation of the specified use cases:

Specifies the test file path
- pytest /path/to/test/file.py
Specify the test class
- pytest /path/to/test/file.py:TestCase
Specify test methods
- pytest another.test::TestClass::test_method
Specifies the test function
- pytest /path/to/test/file.py:test_function

Fourth, the test fixture (Fixtures)

pytestThe test fixture and unittest, nose, nose2of different styles, it can not only achieve setUpand tearDownthis pre-testing and clean-up logic, but also a lot of other powerful features.

4.1 declaration and use

pytestThe test fixture is more like test resources, you only need to define a fixture, then it can be used directly in the use examples. Thanks to pytestthe dependency injection mechanism, you do not need to pass from xx import xxin the form of display to import only need to specify the same name as the parameter to the function of the test parameters, such as:

import pytest


@pytest.fixture
def smtp_connection():
    import smtplib

    return smtplib.SMTP("smtp.gmail.com", 587, timeout=5)


def test_ehlo(smtp_connection):
    response, msg = smtp_connection.ehlo()
    assert response == 250

In the above example defines a test fixture smtp_connection, the test functions test_ehlodefined parameters of the same name in the signature, the pytestframe is automatically injected into the variable.

4.2 Share

In pytest, the multiple test cases can be a test fixture with a plurality of test file sharing. Simply package (Package) defined in the conftest.pydocument, and defines the test fixture written in the file, then the packet within all modules (Module) may be used in all the test cases conftest.pyin the test fixture as defined herein.

For example, if the following file structure test_1/conftest.pydefines the test jig, then test_a.py, and test_b.pyyou can use the test fixture; and test_c.pycan not be used.

`-- test_1
|   |-- conftest.py
|   `-- test_a.py
|   `-- test_b.py
`-- test_2
    `-- test_c.py

4.3 take effect level

unittestAnd nosesupport pre-testing and cleanup of entry into force level: test methods, test classes and test modules.

pytestThe test fixture is also support a variety of levels to take effect, and more abundant. By pytest.fixture specify the scopeset parameters:

function - before the function level, that calls each test function, will regenerate fixture
class - class level, before calling each test class, will regenerate fixture
module - the module level, before loading each test module, will regenerate fixture
package - package level, before loading each package, will regenerate fixture
session - before the session level, to run all use cases, only generated once fixture

When we specify take effect level module level, for example:

import pytest
import smtplib


@pytest.fixture(scope="module")
def smtp_connection():
    return smtplib.SMTP("smtp.gmail.com", 587, timeout=5)

4.4 Pre-testing and clean-up

pytestThe test fixture can be realized pre-testing and cleaning through yieldto split the two logical statements, writing becomes very simple, such as:

import smtplib
import pytest


@pytest.fixture(scope="module")
def smtp_connection():
    smtp_connection = smtplib.SMTP("smtp.gmail.com", 587, timeout=5)
    yield smtp_connection  # provide the fixture value
    print("teardown smtp")
    smtp_connection.close()

In the above example, yield smtp_connectionand the previous statement is equivalent to the pre-test, the yieldreturned resources prepared test smtp_connection;
while the later statement will be executed at the end with the embodiment (to be exact end effective level when the lifecycle test fixture) after execution, equivalent test cleanup.

If the generation of test resources (such as in the example smtp_connection) process support withstatement, you can also write a simpler form:

@pytest.fixture(scope="module")
def smtp_connection():
    with smtplib.SMTP("smtp.gmail.com", 587, timeout=5) as smtp_connection:
        yield smtp_connection  # provide the fixture value

pytestThe paper introduces the test fixture addition to these functions, as well as the parameters of the fixture , factory clamps , using a jig in the jig and other higher-order play more details, please read "pytest fixtures: Explicit, Modular, Scalable" .

Fifth, and is expected to skip the test fails

pytestIn addition to supporting unittestand nosetestskip the test and is expected to fail the way, but also pytest.markprovides a corresponding method:

By skip decorative or pytest.skip skip direct function test
By skipif skipping conditional test
By xfail test is expected to fail

Examples are as follows:

@pytest.mark.skip(reason="no way of currently testing this")
def test_mark_skip():
    ...

def test_skip():
    if not valid_config():
        pytest.skip("unsupported configuration")

@pytest.mark.skipif(sys.version_info < (3, 6), reason="requires python3.6 or higher")
def test_mark_skip_if():
    ...

@pytest.mark.xfail
def test_mark_xfail():
    ...

More games are played on and is expected to skip the test fails, see "Skip and xfail: dealing with tests that can not succeed"

Six sub-test / parametric test

pytestIn addition to supporting unittestthe TestCase.subTest, also supports a more flexible test written in a way the child, that is 参数化测试, through the pytest.mark.parametrizerealization of a decorator.

In the following example, define a test_evaltest function by pytest.mark.parametrizethe specified set of parameters decorators 3, three sub-tests will be generated:

@pytest.mark.parametrize("test_input,expected", [("3+5", 8), ("2+4", 6), ("6*9", 42)])
def test_eval(test_input, expected):
    assert eval(test_input) == expected

The last example deliberately set of parameters lead to failure, run rich with examples you can see the output of the test results:

========================================= test session starts =========================================
platform darwin -- Python 3.7.1, pytest-4.0.1, py-1.7.0, pluggy-0.8.0
rootdir: /Users/prodesire/projects/tests, inifile:
plugins: cov-2.6.0
collected 3 items

test.py ..F                                                                                     [100%]

============================================== FAILURES ===============================================
__________________________________________ test_eval[6*9-42] __________________________________________

test_input = '6*9', expected = 42

    @pytest.mark.parametrize("test_input,expected", [("3+5", 8), ("2+4", 6), ("6*9", 42)])
    def test_eval(test_input, expected):
>       assert eval(test_input) == expected
E       AssertionError: assert 54 == 42
E        +  where 54 = eval('6*9')

test.py:6: AssertionError
================================= 1 failed, 2 passed in 0.09 seconds ==================================

If parameters changed pytest.param, we can also have a higher order of play, such as know the last set of parameters is a failure, so mark it as xfail:

@pytest.mark.parametrize(
    "test_input,expected",
    [("3+5", 8), ("2+4", 6), pytest.param("6*9", 42, marks=pytest.mark.xfail)],
)
def test_eval(test_input, expected):
    assert eval(test_input) == expected

If the value of multiple parameters of test functions hope permutation and combination with each other, so we can write:

@pytest.mark.parametrize("x", [0, 1])
@pytest.mark.parametrize("y", [2, 3])
def test_foo(x, y):
    pass

In the above-described example will be, respectively x=0/y=2, x=1/y=2, x=0/y=3and x=1/y=3into the test functions, considered four test performed.

Seven test result output

pytestThe test results are compared to the output unittestand nosericher, its advantages:

Highlight output, will not be distinguished by different colors or by
Richer context information, the context output code and automatically variable information
Tests show progress
Test results output layout more user-friendly and easy to read

Eight, plug-in system

pytestThe plug-in is very rich, and plug and play, as users do not need to write additional code. About the use of plug-ins, see "Installing and the Using plugins" .

In addition, thanks to pytestgood architecture design and hook mechanism, it becomes easy to write plug-ins to get started. Write about plug-ins, see "Writing plugins" .

Nine, summary

About three Python testing frameworks introduced here would winding down. Write so much, afraid Tell me what you see is tired. We may wish to list a horizontal comparison table summarizes the similarities and differences to the frame unit testing:

	unittest	nose	nose2	pytest
Auto discovery use cases	✔	✔	✔	✔
Specified (all levels) with Example execution	✔	✔	✔	✔
Support assert assertion	weak	weak	weak	Strong
Test Fixture	✔	✔	✔	✔
The type of test fixture	Front and cleanup	Front and cleanup	Front and cleanup	Front, clean up, all kinds of built-in fixtures, all kinds of custom fixtures
Test fixture into effect level	Methods, classes, modules	Methods, classes, modules	Methods, classes, modules	Methods, classes, modules, packages, session
Support is expected to skip the test and fail	✔	✔	✔	✔
Subtest	✔	✔	✔	✔
Test results output	general	better	better	it is good
Plug	-	Rich	general	rich
hook	-	-	✔	✔
Community Ecology	As a standard library, maintained by the official	Stop Maintenance	Maintenance, low activity	Maintenance, active high

Python unit testing framework seemingly wide range of generations of evolution but in reality is, there is a pattern. Seize its characteristics, combined with the scene, you can easily make a choice.

If you do not want or are not allowed to install third-party libraries, it unittestis the best and only choice. Conversely, pytestis the best choice, many Python open source projects (such as the famous Requests ) are used pytestas the unit testing framework. Indeed, even nose2in official documents on both suggest that you use pytest, this is too much admiration ah!

"Explain open source projects Series" - to let people interested in open source projects are no longer afraid, let sponsors open source projects are no longer alone. Follow our articles, you'll discover the fun of programming, the use of open source projects and found to be involved so simple. Welcome messages to contact us, join us, so that more people fall in love with open source, open source contribution ~