Python's must-have unit testing framework - unittest

To build an automated testing framework with Python, we need to organize use cases and test execution. Here bloggers recommend Python's standard library - unittest.

unittest is a member of the xUnit family of frameworks. If you know the other members of xUnit, it should be easy for you to use unittest. They all work in the same way.

How unittest works

The four core concepts in unittest are: test case, test suite, test runner, test fixture .

Let's explain the meaning of these four concepts separately. Let's first look at a static class diagram of unittest (the following class diagram and explanation are all from the Internet, the original link ):

unittest class diagram

An instance of TestCase is a test case. What is a test case? It is a complete test process, including the setup of the pre-test preparation environment (setUp), the execution of the test code (run), and the restoration of the post-test environment (tearDown). The essence of unit test is here. A test case is a complete test unit. By running this test unit, a certain problem can be verified.

A collection of multiple test cases is called TestSuite, and TestSuite can also nest TestSuite.

TestLoader is used to load TestCases into TestSuite. There are several loadTestsFrom__() methods, which are to find TestCases from various places, create their instances, add them to TestSuite, and return a TestSuite instance.

TextTestRunner is used to execute test cases, in which run(test) will execute the run(result) method in TestSuite/TestCase.
The results of the test will be saved to the TextTestResult instance, including how many test cases were run, how many were successful, and how many failed.

The construction and destruction of a test case environment is a fixture.

A class that inherits unittest.TestCase is a test case, but if there are multiple test methods that start with, then for each such method, a TestCase instance will be generated when loading, such as: a class has Four test_xxx methods, and finally four test cases when loaded into the suite.

Here the whole process is clear:

Write TestCase, then load TestCase to TestSuite by TestLoader, and then run TestSuite by TextTestRunner. The result of running is saved in TextTestResult. When we execute it through the command line or unittest.main(), main will call run in TextTestRunner to execute, Or we can execute the use case directly through TextTestRunner. Here's a note, when the Runner is executed, the execution result is output to the console by default, we can set it to output to a file and view the results in the file (you may have heard of HTMLTestRunner, yes, it can output the results to In HTML, a beautiful report is generated, which is the same as TextTestRunner, as can be seen from the name, we will talk about this later).

unittest instance

Let's get a better understanding of unittest through some examples.

Let's first prepare some methods to be tested:

mathfunc.py

def add(a, b):
    return a+b

def minus(a, b):
    return a-b

def multi(a, b):
    return a*b

def divide(a, b):
    return a/b

Simple example

Next we write a test for these methods:

test_mathfunc.py

# -*- coding: utf-8 -*-

import unittest
from mathfunc import *


class TestMathFunc(unittest.TestCase):
    """Test mathfuc.py"""

    def test_add(self):
        """Test method add(a, b)"""
        self.assertEqual(3, add(1, 2))
        self.assertNotEqual(3, add(2, 2))

    def test_minus(self):
        """Test method minus(a, b)"""
        self.assertEqual(1, minus(3, 2))

    def test_multi(self):
        """Test method multi(a, b)"""
        self.assertEqual(6, multi(2, 3))

    def test_divide(self):
        """Test method divide(a, b)"""
        self.assertEqual(2, divide(6, 3))
        self.assertEqual(2.5, divide(5, 2))

if __name__ == '__main__':
    unittest.main()

Results of the:

.F..
======================================================================
FAIL: test_divide (__main__.TestMathFunc)
Test method divide(a, b)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:/py/test_mathfunc.py", line 26, in test_divide
    self.assertEqual(2.5, divide(5, 2))
AssertionError: 2.5 != 2

----------------------------------------------------------------------
Ran 4 tests in 0.000s

FAILED (failures=1)

It can be seen that a total of 4 tests were run, 1 failed, and the reason for the failure was given, which 2.5 != 2 means that there is a problem with our divide method.

This is a simple test with a few points to note:

The first line gives the identification of the result of each use case execution, success is ., failure is F, error is E, skip is S. It can also be seen from the above that the execution of the test has nothing to do with the order of the methods. test_divide is written in the fourth, but it is executed in the second.
Each test method test starts with , otherwise it is not recognized by unittest.
Adding parameters to unittest.main() verbosity can control the detail level of the output error report. The default is 1, if set to 0, the execution result of each use case will not be output, that is, there is no line 1 in the above result; if set to 2, the detailed execution results are output as follows:

test_add (__main__.TestMathFunc)
Test method add(a, b) ... ok
test_divide (__main__.TestMathFunc)
Test method divide(a, b) ... FAIL
test_minus (__main__.TestMathFunc)
Test method minus(a, b) ... ok
test_multi (__main__.TestMathFunc)
Test method multi(a, b) ... ok

======================================================================
FAIL: test_divide (__main__.TestMathFunc)
Test method divide(a, b)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:/py/test_mathfunc.py", line 26, in test_divide
    self.assertEqual(2.5, divide(5, 2))
AssertionError: 2.5 != 2

----------------------------------------------------------------------
Ran 4 tests in 0.002s

FAILED (failures=1)

It can be seen that the detailed execution of each use case, the use case name, and the use case description are output (add """Doc String""" in the code example under the test method, and the character will be displayed when the use case is executed. String as the description of this use case, adding appropriate comments can make the output test report easier to read )

Organizing TestSuites

The above code example how to write a simple test, but there are two problems, how can we control the order of execution of the test cases? (Several test methods in the example here are not necessarily related, but the use cases you write in the future may have a relationship, you need to execute method A first, and then execute method B), we will use TestSuite. The cases we add to TestSuite are executed in the order in which they were added .

The second question is that we only have one test file now, we can directly execute the file, but if there are multiple test files, how to organize them, we can't execute them one by one, and the answer is also in TestSuite.

Here's an example:

In the folder we create a new file, test_suite.py :

# -*- coding: utf-8 -*-

import unittest
from test_mathfunc import TestMathFunc

if __name__ == '__main__':
    suite = unittest.TestSuite()

    tests = [TestMathFunc("test_add"), TestMathFunc("test_minus"), TestMathFunc("test_divide")]
    suite.addTests(tests)

    runner = unittest.TextTestRunner(verbosity=2)
    runner.run(suite)

Results of the:

test_add (test_mathfunc.TestMathFunc)
Test method add(a, b) ... ok
test_minus (test_mathfunc.TestMathFunc)
Test method minus(a, b) ... ok
test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b) ... FAIL

======================================================================
FAIL: test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\py\test_mathfunc.py", line 26, in test_divide
    self.assertEqual(2.5, divide(5, 2))
AssertionError: 2.5 != 2

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)

As you can see, the execution is as we expected: three cases are executed, and the order is in the order we added them to the suite.

The TestSuite addTests() method is used above and the TestCase list is passed in directly. We can also:

# 直接用addTest方法添加单个TestCase
suite.addTest(TestMathFunc("test_multi"))

# 用addTests + TestLoader
# loadTestsFromName()，传入'模块名.TestCase名'
suite.addTests(unittest.TestLoader().loadTestsFromName('test_mathfunc.TestMathFunc'))
suite.addTests(unittest.TestLoader().loadTestsFromNames(['test_mathfunc.TestMathFunc']))  # loadTestsFromNames()，类似，传入列表

# loadTestsFromTestCase()，传入TestCase
suite.addTests(unittest.TestLoader().loadTestsFromTestCase(TestMathFunc))

Note that the case cannot be sorted by the TestLoader method. At the same time, the suite can also be set in the suite.

output the result to a file

The use case is organized, but the results can only be output to the console, so there is no way to view the previous execution records, we want to output the results to a file. Very simple, see example:

Modify test_suite.py :

# -*- coding: utf-8 -*-

import unittest
from test_mathfunc import TestMathFunc

if __name__ == '__main__':
    suite = unittest.TestSuite()
    suite.addTests(unittest.TestLoader().loadTestsFromTestCase(TestMathFunc))

    with open('UnittestTextReport.txt', 'a') as f:
        runner = unittest.TextTestRunner(stream=f, verbosity=2)
        runner.run(suite)

Execute this file, you can see that UnittestTextReport.txt is generated in the same directory , and all execution reports are output to this file. Now we have a test report in txt format.

test fixture之setUp() tearDown()

The entire test above basically ran, but there may be some special cases: what if my test needs to prepare the environment before each execution, or needs to do some cleanup after each execution? For example, you need to connect to the database before execution, and after the execution is complete, you need to restore data and disconnect. It is impossible to add code to prepare the environment and clean up the environment in every test method.

This involves the test fixture we said before, modify test_mathfunc.py :

# -*- coding: utf-8 -*-

import unittest
from mathfunc import *


class TestMathFunc(unittest.TestCase):
    """Test mathfuc.py"""

    def setUp(self):
        print "do something before test.Prepare environment."

    def tearDown(self):
        print "do something after test.Clean up."

    def test_add(self):
        """Test method add(a, b)"""
        print "add"
        self.assertEqual(3, add(1, 2))
        self.assertNotEqual(3, add(2, 2))

    def test_minus(self):
        """Test method minus(a, b)"""
        print "minus"
        self.assertEqual(1, minus(3, 2))

    def test_multi(self):
        """Test method multi(a, b)"""
        print "multi"
        self.assertEqual(6, multi(2, 3))

    def test_divide(self):
        """Test method divide(a, b)"""
        print "divide"
        self.assertEqual(2, divide(6, 3))
        self.assertEqual(2.5, divide(5, 2))

We added setUp() and tearDown() two methods (actually overriding these two methods of TestCase), these two methods are executed once before and after each test method is executed, setUp is used to prepare the environment for the test, tearDown is used to clean up environment, ready for later testing.

Let's do it again:

test_add (test_mathfunc.TestMathFunc)
Test method add(a, b) ... ok
test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b) ... FAIL
test_minus (test_mathfunc.TestMathFunc)
Test method minus(a, b) ... ok
test_multi (test_mathfunc.TestMathFunc)
Test method multi(a, b) ... ok

======================================================================
FAIL: test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\py\test_mathfunc.py", line 36, in test_divide
    self.assertEqual(2.5, divide(5, 2))
AssertionError: 2.5 != 2

----------------------------------------------------------------------
Ran 4 tests in 0.000s

FAILED (failures=1)
do something before test.Prepare environment.
add
do something after test.Clean up.
do something before test.Prepare environment.
divide
do something after test.Clean up.
do something before test.Prepare environment.
minus
do something after test.Clean up.
do something before test.Prepare environment.
multi
do something after test.Clean up.

You can see that setUp and tearDown are executed once before and after each case is executed.

In this way, an instance of the class is created for each test case through the setup method each time, and the instance is closed after each test method ends. Can each test method use a test instance instead of creating a new one every time? , through the @classmethod method, if you want to prepare the environment before all cases are executed, and clean up the environment after all cases are executed, we can use setUpClass() and tearDownClass(), these two can initialize data at the class level, Replaces method-level initialization so that each test method can share this initialization data

...

class TestMathFunc(unittest.TestCase):
    """Test mathfuc.py"""

    @classmethod
    def setUpClass(cls):
        print "This setUpClass() method only called once."

    @classmethod
    def tearDownClass(cls):
        print "This tearDownClass() method only called once too."

...

The execution result is as follows:

...
This setUpClass() method only called once.
do something before test.Prepare environment.
add
do something after test.Clean up.
...
do something before test.Prepare environment.
multi
do something after test.Clean up.
This tearDownClass() method only called once too.

You can see that both setUpClass and tearDownClass are executed only once.

skip a case

What if we temporarily want to skip a case and not execute it? unittest also provides several methods:

skip decorator

...

class TestMathFunc(unittest.TestCase):
    """Test mathfuc.py"""

    ...

    @unittest.skip("I don't want to run this case.")
    def test_divide(self):
        """Test method divide(a, b)"""
        print "divide"
        self.assertEqual(2, divide(6, 3))
        self.assertEqual(2.5, divide(5, 2))

implement:

...
test_add (test_mathfunc.TestMathFunc)
Test method add(a, b) ... ok
test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b) ... skipped "I don't want to run this case."
test_minus (test_mathfunc.TestMathFunc)
Test method minus(a, b) ... ok
test_multi (test_mathfunc.TestMathFunc)
Test method multi(a, b) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK (skipped=1)

You can see that the total number of tests is still 4, but the divide() method is skipped.

There are three skip decorators unittest.skip(reason), unittest.skipIf(condition, reason), unittest.skipUnless(condition, reason), skip unconditionally skip, skipIf skips when condition is True, skipUnless skips when condition is False.

TestCase.skipTest() method

...

class TestMathFunc(unittest.TestCase):
    """Test mathfuc.py"""

    ...

    def test_divide(self):
        """Test method divide(a, b)"""
        self.skipTest('Do not run this.')
        print "divide"
        self.assertEqual(2, divide(6, 3))
        self.assertEqual(2.5, divide(5, 2))

output:

...
test_add (test_mathfunc.TestMathFunc)
Test method add(a, b) ... ok
test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b) ... skipped 'Do not run this.'
test_minus (test_mathfunc.TestMathFunc)
Test method minus(a, b) ... ok
test_multi (test_mathfunc.TestMathFunc)
Test method multi(a, b) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK (skipped=1)

The effect is the same as the above decorator, skipping the divide method.

Advanced - use HTMLTestRunner to output beautiful HTML reports

We can output the text execution report in txt format, but the text report is too rudimentary. Do you want a higher HTML report? But unittest itself does not have HTML reports, we can only resort to external libraries.

HTMLTestRunner is a third-party unittest HTML report library. First, we download HTMLTestRunner.py and put it in the current directory, or under your 'C:\Python27\Lib', and you can import and run it.

download link:

Official original: http://tungwaiyip.info/software/HTMLTestRunner.html

Gray-blue modified version: HTMLTestRunner.py (adjusted format, Chinese display)

Modify our test_suite.py:

# -*- coding: utf-8 -*-

import unittest
from test_mathfunc import TestMathFunc
from HTMLTestRunner import HTMLTestRunner

if __name__ == '__main__':
    suite = unittest.TestSuite()
    suite.addTests(unittest.TestLoader().loadTestsFromTestCase(TestMathFunc))

    with open('HTMLReport.html', 'w') as f:
        runner = HTMLTestRunner(stream=f,
                                title='MathFunc Test Report',
                                description='generated by HTMLTestRunner.',
                                verbosity=2
                                )
        runner.run(suite)

In this way, during execution, we can see the execution in the console, as follows:

ok test_add (test_mathfunc.TestMathFunc)
F  test_divide (test_mathfunc.TestMathFunc)
ok test_minus (test_mathfunc.TestMathFunc)
ok test_multi (test_mathfunc.TestMathFunc)

Time Elapsed: 0:00:00.001000

And output the HTML test report, HTMLReport.html, as shown in the figure:

html report

This time the beautiful HTML report is also available. In fact, you can find that the execution method of HTMLTestRunner is very similar to that of TextTestRunner. You can compare it with my example above, that is, replace the runner in the class diagram with HTMLTestRunner, and display the TestResult in HTML form. If you study enough Deep, you can write your own runner to generate more complex and beautiful reports.

To summarize :

unittest is a unit testing framework that comes with Python, and we can use it as a use case organization and execution framework for our automated testing framework.
The process of unittest: write TestCase, then load TestCase into TestSuite by TestLoader, and then run TestSuite by TextTestRunner. The result of running is saved in TextTestResult. When we execute it through the command line or unittest.main(), main will call the text in TextTestRunner. run to execute, or we can execute the use case directly through TextTestRunner.
A class that inherits unittest.TestCase is a TestCase, in which test the method starting with is loaded as a real TestCase at load time.
The verbosity parameter can control the output of the execution result, which 0 is a simple report, 1 a general report, and 2 a detailed report.
You can add case or suite to the suite through addTest and addTests, you can use the loadTestsFrom__() method of TestLoader.
Use setUp(), tearDown(), setUpClass()and tearDownClass()you can set up the environment before the use case is executed, and clean up the environment after the use case is executed
We can skip a case through the skip, skipIf, skipUnless decorators, or use the TestCase.skipTest method.
Add stream to the parameter to output the report to a file: you can use TextTestRunner to output txt reports, and you can use HTMLTestRunner to output html reports.

We have not discussed the use of the command line and the module-level fixtures here. Interested students can search for materials and learn by themselves.

Reprinted: https://blog.csdn.net/huilan_same/article/details/52944782

For more articles about python selenium, please pay attention to my CSDN column: Detailed explanation of Python Selenium automated testing