Python's must-have unit testing framework - unittest

To build an automated testing framework with Python, we need to organize use cases and test execution. Here bloggers recommend Python's standard library - unittest.

unittest is a member of the xUnit family of frameworks. If you know the other members of xUnit, it should be easy for you to use unittest. They all work in the same way.

How unittest works

The four core concepts in unittest are: test case, test suite, test runner, test fixture .

Let's explain the meaning of these four concepts separately. Let's first look at a static class diagram of unittest (the following class diagram and explanation are all from the Internet, the original link ):

• An instance of TestCase is a test case. What is a test case? It is a complete test process, including the setup of the pre-test preparation environment (setUp), the execution of the test code (run), and the restoration of the post-test environment (tearDown). The essence of unit test is here. A test case is a complete test unit. By running this test unit, a certain problem can be verified.

• A collection of multiple test cases is called TestSuite, and TestSuite can also nest TestSuite.

• TestLoader is used to load TestCases into TestSuite. There are several loadTestsFrom__() methods, which are to find TestCases from various places, create their instances, add them to TestSuite, and return a TestSuite instance.

• TextTestRunner is used to execute test cases, in which run(test) will execute the run(result) method in TestSuite/TestCase. 
  The results of the test will be saved to the TextTestResult instance, including how many test cases were run, how many were successful, and how many failed.

• The construction and destruction of a test case environment is a fixture.

A class that inherits unittest.TestCase is a test case, but if there are multiple  test methods that start with, then for each such method, a TestCase instance will be generated when loading, such as: a class has Four test_xxx methods, and finally four test cases when loaded into the suite.

Here the whole process is clear:

Write TestCase, then load TestCase to TestSuite by TestLoader, and then run TestSuite by TextTestRunner. The result of running is saved in TextTestResult. When we execute it through the command line or unittest.main(), main will call run in TextTestRunner to execute, Or we can execute the use case directly through TextTestRunner. Here's a note, when the Runner is executed, the execution result is output to the console by default, we can set it to output to a file and view the results in the file (you may have heard of HTMLTestRunner, yes, it can output the results to In HTML, a beautiful report is generated, which is the same as TextTestRunner, as can be seen from the name, we will talk about this later).

unittest instance

Let's get a better understanding of unittest through some examples.

Let's first prepare some methods to be tested:

mathfunc.py

def add(a, b):
    return a+b

def minus(a, b):
    return a-b

def multi(a, b):
    return a*b

def divide(a, b):
    return a/b

Simple example

Next we write a test for these methods:

test_mathfunc.py

# -*- coding: utf-8 -*-

import unittest
from mathfunc import *


class TestMathFunc(unittest.TestCase):
    """Test mathfuc.py"""

    def test_add(self):
        """Test method add(a, b)"""
        self.assertEqual(3, add(1, 2))
        self.assertNotEqual(3, add(2, 2))

    def test_minus(self):
        """Test method minus(a, b)"""
        self.assertEqual(1, minus(3, 2))

    def test_multi(self):
        """Test method multi(a, b)"""
        self.assertEqual(6, multi(2, 3))

    def test_divide(self):
        """Test method divide(a, b)"""
        self.assertEqual(2, divide(6, 3))
        self.assertEqual(2.5, divide(5, 2))

if __name__ == '__main__':
    unittest.main()

Results of the:

.F..
======================================================================
FAIL: test_divide (__main__.TestMathFunc)
Test method divide(a, b)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:/py/test_mathfunc.py", line 26, in test_divide
    self.assertEqual(2.5, divide(5, 2))
AssertionError: 2.5 != 2

----------------------------------------------------------------------
Ran 4 tests in 0.000s

FAILED (failures=1)

It can be seen that a total of 4 tests were run, 1 failed, and the reason for the failure was given, which 2.5 != 2 means that there is a problem with our divide method.

This is a simple test with a few points to note:

1. 在第一行给出了每一个用例执行的结果的标识，成功是 .，失败是 F，出错是 E，跳过是 S。从上面也可以看出，测试的执行跟方法的顺序没有关系，test_divide写在了第4个，但是却是第2个执行的。

2. 每个测试方法均以 test 开头，否则是不被unittest识别的。

3. 在unittest.main()中加 verbosity 参数可以控制输出的错误报告的详细程度，默认是 1，如果设为 0，则不输出每一用例的执行结果，即没有上面的结果中的第1行；如果设为 2，则输出详细的执行结果，如下：

test_add (__main__.TestMathFunc)
Test method add(a, b) ... ok
test_divide (__main__.TestMathFunc)
Test method divide(a, b) ... FAIL
test_minus (__main__.TestMathFunc)
Test method minus(a, b) ... ok
test_multi (__main__.TestMathFunc)
Test method multi(a, b) ... ok

======================================================================
FAIL: test_divide (__main__.TestMathFunc)
Test method divide(a, b)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:/py/test_mathfunc.py", line 26, in test_divide
    self.assertEqual(2.5, divide(5, 2))
AssertionError: 2.5 != 2

----------------------------------------------------------------------
Ran 4 tests in 0.002s

FAILED (failures=1)

可以看到，每一个用例的详细执行情况以及用例名，用例描述均被输出了出来（在测试方法下加代码示例中的”“”Doc String”“”，在用例执行时，会将该字符串作为此用例的描述，加合适的注释能够使输出的测试报告更加便于阅读）

组织TestSuite

上面的代码示例了如何编写一个简单的测试，但有两个问题，我们怎么控制用例执行的顺序呢？（这里的示例中的几个测试方法并没有一定关系，但之后你写的用例可能会有先后关系，需要先执行方法A，再执行方法B），我们就要用到TestSuite了。我们添加到TestSuite中的case是会按照添加的顺序执行的。

问题二是我们现在只有一个测试文件，我们直接执行该文件即可，但如果有多个测试文件，怎么进行组织，总不能一个个文件执行吧，答案也在TestSuite中。

下面来个例子：

在文件夹中我们再新建一个文件，test_suite.py：

# -*- coding: utf-8 -*-

import unittest
from test_mathfunc import TestMathFunc

if __name__ == '__main__':
    suite = unittest.TestSuite()

    tests = [TestMathFunc("test_add"), TestMathFunc("test_minus"), TestMathFunc("test_divide")]
    suite.addTests(tests)

    runner = unittest.TextTestRunner(verbosity=2)
    runner.run(suite)

执行结果：

test_add (test_mathfunc.TestMathFunc)
Test method add(a, b) ... ok
test_minus (test_mathfunc.TestMathFunc)
Test method minus(a, b) ... ok
test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b) ... FAIL

======================================================================
FAIL: test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\py\test_mathfunc.py", line 26, in test_divide
    self.assertEqual(2.5, divide(5, 2))
AssertionError: 2.5 != 2

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)

可以看到，执行情况跟我们预料的一样：执行了三个case，并且顺序是按照我们添加进suite的顺序执行的。

上面用了TestSuite的 addTests() 方法，并直接传入了TestCase列表，我们还可以：

# 直接用addTest方法添加单个TestCase
suite.addTest(TestMathFunc("test_multi"))

# 用addTests + TestLoader
# loadTestsFromName()，传入'模块名.TestCase名'
suite.addTests(unittest.TestLoader().loadTestsFromName('test_mathfunc.TestMathFunc'))
suite.addTests(unittest.TestLoader().loadTestsFromNames(['test_mathfunc.TestMathFunc']))  # loadTestsFromNames()，类似，传入列表

# loadTestsFromTestCase()，传入TestCase
suite.addTests(unittest.TestLoader().loadTestsFromTestCase(TestMathFunc))

注意，用TestLoader的方法是无法对case进行排序的，同时，suite中也可以套suite。

将结果输出到文件中

用例组织好了，但结果只能输出到控制台，这样没有办法查看之前的执行记录，我们想将结果输出到文件。很简单，看示例：

修改test_suite.py：

# -*- coding: utf-8 -*-

import unittest
from test_mathfunc import TestMathFunc

if __name__ == '__main__':
    suite = unittest.TestSuite()
    suite.addTests(unittest.TestLoader().loadTestsFromTestCase(TestMathFunc))

    with open('UnittestTextReport.txt', 'a') as f:
        runner = unittest.TextTestRunner(stream=f, verbosity=2)
        runner.run(suite)

执行此文件，可以看到，在同目录下生成了UnittestTextReport.txt，所有的执行报告均输出到了此文件中，这下我们便有了txt格式的测试报告了。

test fixture之setUp() tearDown()

上面整个测试基本跑了下来，但可能会遇到点特殊的情况：如果我的测试需要在每次执行之前准备环境，或者在每次执行完之后需要进行一些清理怎么办？比如执行前需要连接数据库，执行完成之后需要还原数据、断开连接。总不能每个测试方法中都添加准备环境、清理环境的代码吧。

这就要涉及到我们之前说过的test fixture了，修改test_mathfunc.py：

# -*- coding: utf-8 -*-

import unittest
from mathfunc import *


class TestMathFunc(unittest.TestCase):
    """Test mathfuc.py"""

    def setUp(self):
        print "do something before test.Prepare environment."

    def tearDown(self):
        print "do something after test.Clean up."

    def test_add(self):
        """Test method add(a, b)"""
        print "add"
        self.assertEqual(3, add(1, 2))
        self.assertNotEqual(3, add(2, 2))

    def test_minus(self):
        """Test method minus(a, b)"""
        print "minus"
        self.assertEqual(1, minus(3, 2))

    def test_multi(self):
        """Test method multi(a, b)"""
        print "multi"
        self.assertEqual(6, multi(2, 3))

    def test_divide(self):
        """Test method divide(a, b)"""
        print "divide"
        self.assertEqual(2, divide(6, 3))
        self.assertEqual(2.5, divide(5, 2))

我们添加了 setUp() 和 tearDown() 两个方法（其实是重写了TestCase的这两个方法），这两个方法在每个测试方法执行前以及执行后执行一次，setUp用来为测试准备环境，tearDown用来清理环境，已备之后的测试。

我们再执行一次：

test_add (test_mathfunc.TestMathFunc)
Test method add(a, b) ... ok
test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b) ... FAIL
test_minus (test_mathfunc.TestMathFunc)
Test method minus(a, b) ... ok
test_multi (test_mathfunc.TestMathFunc)
Test method multi(a, b) ... ok

======================================================================
FAIL: test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\py\test_mathfunc.py", line 36, in test_divide
    self.assertEqual(2.5, divide(5, 2))
AssertionError: 2.5 != 2

----------------------------------------------------------------------
Ran 4 tests in 0.000s

FAILED (failures=1)
do something before test.Prepare environment.
add
do something after test.Clean up.
do something before test.Prepare environment.
divide
do something after test.Clean up.
do something before test.Prepare environment.
minus
do something after test.Clean up.
do something before test.Prepare environment.
multi
do something after test.Clean up.

可以看到setUp和tearDown在每次执行case前后都执行了一次。

这样每次通过setup方法为每个测试用例创建一个类的实例，并且自每个测试方法结束之后进行关闭实例，能否让各个测试方法工用一个测试实例呢，而不是每次都创建一个新的测试实例，通过@classmethod方法，如果想要在所有case执行之前准备一次环境，并在所有case执行结束之后再清理环境，我们可以用 setUpClass() 与 tearDownClass(),这两个放阿飞可以在类级别来初始化数据，替代了方法级别的初始化，这样各个测试方法就可以共享这些初始化数据

...

class TestMathFunc(unittest.TestCase):
    """Test mathfuc.py"""

    @classmethod
    def setUpClass(cls):
        print "This setUpClass() method only called once."

    @classmethod
    def tearDownClass(cls):
        print "This tearDownClass() method only called once too."

...

执行结果如下：

...
This setUpClass() method only called once.
do something before test.Prepare environment.
add
do something after test.Clean up.
...
do something before test.Prepare environment.
multi
do something after test.Clean up.
This tearDownClass() method only called once too.

可以看到setUpClass以及tearDownClass均只执行了一次。

跳过某个case

如果我们临时想要跳过某个case不执行怎么办？unittest也提供了几种方法：

1. skip装饰器

...

class TestMathFunc(unittest.TestCase):
    """Test mathfuc.py"""

    ...

    @unittest.skip("I don't want to run this case.")
    def test_divide(self):
        """Test method divide(a, b)"""
        print "divide"
        self.assertEqual(2, divide(6, 3))
        self.assertEqual(2.5, divide(5, 2))

执行：

...
test_add (test_mathfunc.TestMathFunc)
Test method add(a, b) ... ok
test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b) ... skipped "I don't want to run this case."
test_minus (test_mathfunc.TestMathFunc)
Test method minus(a, b) ... ok
test_multi (test_mathfunc.TestMathFunc)
Test method multi(a, b) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK (skipped=1)

可以看到总的test数量还是4个，但divide()方法被skip了。

skip装饰器一共有三个 unittest.skip(reason)、unittest.skipIf(condition, reason)、unittest.skipUnless(condition, reason)，skip无条件跳过，skipIf当condition为True时跳过，skipUnless当condition为False时跳过。

1. TestCase.skipTest()方法

...

class TestMathFunc(unittest.TestCase):
    """Test mathfuc.py"""

    ...

    def test_divide(self):
        """Test method divide(a, b)"""
        self.skipTest('Do not run this.')
        print "divide"
        self.assertEqual(2, divide(6, 3))
        self.assertEqual(2.5, divide(5, 2))

输出：

...
test_add (test_mathfunc.TestMathFunc)
Test method add(a, b) ... ok
test_divide (test_mathfunc.TestMathFunc)
Test method divide(a, b) ... skipped 'Do not run this.'
test_minus (test_mathfunc.TestMathFunc)
Test method minus(a, b) ... ok
test_multi (test_mathfunc.TestMathFunc)
Test method multi(a, b) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK (skipped=1)

效果跟上面的装饰器一样，跳过了divide方法。

进阶——用HTMLTestRunner输出漂亮的HTML报告

我们能够输出txt格式的文本执行报告了，但是文本报告太过简陋，是不是想要更加高大上的HTML报告？但unittest自己可没有带HTML报告，我们只能求助于外部的库了。

HTMLTestRunner是一个第三方的unittest HTML报告库，首先我们下载HTMLTestRunner.py，并放到当前目录下，或者你的’C:\Python27\Lib’下，就可以导入运行了。

下载地址：

官方原版：http://tungwaiyip.info/software/HTMLTestRunner.html

灰蓝修改版：HTMLTestRunner.py(已调整格式，中文显示）

修改我们的 test_suite.py：

# -*- coding: utf-8 -*-

import unittest
from test_mathfunc import TestMathFunc
from HTMLTestRunner import HTMLTestRunner

if __name__ == '__main__':
    suite = unittest.TestSuite()
    suite.addTests(unittest.TestLoader().loadTestsFromTestCase(TestMathFunc))

    with open('HTMLReport.html', 'w') as f:
        runner = HTMLTestRunner(stream=f,
                                title='MathFunc Test Report',
                                description='generated by HTMLTestRunner.',
                                verbosity=2
                                )
        runner.run(suite)

这样，在执行时，在控制台我们能够看到执行情况，如下：

ok test_add (test_mathfunc.TestMathFunc)
F  test_divide (test_mathfunc.TestMathFunc)
ok test_minus (test_mathfunc.TestMathFunc)
ok test_multi (test_mathfunc.TestMathFunc)

Time Elapsed: 0:00:00.001000

并且输出了HTML测试报告，HTMLReport.html，如图：

这下漂亮的HTML报告也有了。其实你能发现，HTMLTestRunner的执行方法跟TextTestRunner很相似，你可以跟我上面的示例对比一下，就是把类图中的runner换成了HTMLTestRunner，并将TestResult用HTML的形式展现出来，如果你研究够深，可以写自己的runner，生成更复杂更漂亮的报告。

总结一下：

1. unittest是Python自带的单元测试框架，我们可以用其来作为我们自动化测试框架的用例组织执行框架。
2. unittest的流程：写好TestCase，然后由TestLoader加载TestCase到TestSuite，然后由TextTestRunner来运行TestSuite，运行的结果保存在TextTestResult中，我们通过命令行或者unittest.main()执行时，main会调用TextTestRunner中的run来执行，或者我们可以直接通过TextTestRunner来执行用例。
3. 一个class继承unittest.TestCase即是一个TestCase，其中以 test 开头的方法在load时被加载为一个真正的TestCase。
4. verbosity参数可以控制执行结果的输出，0 是简单报告、1 是一般报告、2 是详细报告。
5. You can add case or suite to the suite through addTest and addTests, you can use the loadTestsFrom__() method of TestLoader.
6. Use  setUp(), tearDown(), setUpClass()and  tearDownClass()you can set up the environment before the use case is executed, and clean up the environment after the use case is executed
7. We can skip a case through the skip, skipIf, skipUnless decorators, or use the TestCase.skipTest method.
8. Add stream to the parameter to output the report to a file: you can use TextTestRunner to output txt reports, and you can use HTMLTestRunner to output html reports.

We have not discussed the use of the command line and module-level fixtures here. Interested students can search for materials and learn by themselves.

Reprinted: https://blog.csdn.net/huilan_same/article/details/52944782

---

For more articles about python selenium, please pay attention to my CSDN column: Detailed explanation of Python Selenium automated testing