1 Overview

The testing methodology of the procedure is based on the points mentioned in the previous article:

Functional testing is not recommended for automation
Interface testing is the most cost-effective
Interface testing can be automated

The test automation mentioned later will also be introduced around the interface automation .

If you want to learn automated testing, here I recommend a set of videos for you. This video can be said to be the number one automated testing tutorial on the entire network at station B. At the same time, the number of people online has reached 1,000, and there are notes to collect and share with you. Dashen Technical Exchange: 798478386

The most detailed collection of practical tutorials for automated testing of Python interfaces (the latest version in actual combat) taught by station B Including: 1. Why interface automation is needed, 2. The overall view of interface automation requests, 3. Interface automation, interface combat, etc. For more exciting videos, please pay attention to the UP account. https://www.bilibili.com/video/BV17p4y1B77x/?spm_id_from=333.337.search-card.all.click

The test language chosen for this series is the python scripting language. Since its official document has a relatively clear explanation of the principle, this article will not do some redundant translation work. It is biased towards the actual combat part, and in order to be biased toward actual combat, it will also be explained in combination with IDE tools and project organization.

The reasons are as follows:

Scripting language, extremely efficient in development and iteration
There are many third-party extension libraries, and there are many ready-made tools that I can use

Before officially entering the field of automated testing , we must first establish such values. In a publication on software testing by Google's internal engineers it is mentioned:

"Automated testing of software is costly, and the cost is not low. It is basically equivalent to establishing a parallel test development project on the basis of the original function development project ."

In other words, if you have your expectations for automated testing, then you must pay the corresponding price and energy. Good things also require excellent people to spend a lot of time to complete.

2 PyUnit testing framework

Using python as an automated programming language, it is natural to use pyunit as an automated testing framework.

The content of the following part is mainly from the official documentation of pyunit. This article only makes some translations and simple structural adjustments. This part belongs to the basic principles and concepts of the test framework, and it is necessary to understand it before writing code.

Python's unit testing framework PyUnit can be considered as the Python language implementation version of the unit testing framework JUnit under the Java language, and even one of its authors, Kent Beck , is the author of JUnit.

unittest should achieve the following goals:

Support automated testing
Let all test scripts share the setup and shutdown code
Test case scripts can be organized in collections
Separate all test scripts from the test reporting framework

In order to achieve the above goals, unittest supports the following important concepts:

test fixture

Do some preparation work for one or more test cases, for example: connect to a database, create a directory, or start a process
test case

A test case is the smallest unit of test behavior, and the test check is performed by comparing some input and output values
test suite

A collection that aggregates and organizes test cases or test case collections . All test cases in a test suite can be executed in batches
test runner

The component that organizes the execution of test scripts. The test executor displays the test results of the test script through some graphical interfaces, text interfaces or by returning some special values. Mainly used to generate test reports

3 Basic example

The following example is also from the official document basic_demo.py :

# coding:utf-8
"""
基本的自动化测试脚本 basic_demo.py
"""
__author__ = 'zheng'
 
import unittest
 
 
class TestStringMethods(unittest.TestCase):
 
    def setUp(self):
        print 'init by setUp...'
 
    def tearDown(self):
        print 'end by tearDown...'
 
    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')
 
    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())
        self.assertTrue('Foo'.isupper())
 
    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)
 
 
if __name__ == '__main__':
    unittest.main()

Although the official documentation describes several ways to organize test case scripts:

independent test function
Single use case test class
Multiple use case test class

Different writing forms will have different organization methods. For details, please refer to the official documents. After studying the official documents, the author of this article likes the third method of multi-use case test class , which is the method of the basic example above. This method has the following characteristics:

The test class inherits from unittest.TestCase
A test class can manage multiple test script functions
Test script function names need to start with test_
All test functions in a test class share the setUp and tearDown functions

Run this program in console:

➜  src git:(master) ✗ python basic_demo.py
init by setUp...
Fend by tearDown...
init by setUp...
end by tearDown...
.init by setUp...
end by tearDown...
.
======================================================================
FAIL: test_isupper (__main__.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "basic_demo.py", line 24, in test_isupper
    self.assertTrue('Foo'.isupper())
AssertionError: False is not true
 
----------------------------------------------------------------------
Ran 3 tests in 0.001s
 
FAILED (failures=1)
➜  src git:(master) ✗

The main function of the previous basic example adopts the simplest way to directly run all test cases and generate a default text report. In fact, it only needs to make some simple modifications to the calling function, so that these test cases can be organized reasonably, and useful data information can be obtained, so as to integrate with the information system and form a better extension.

if __name__ == '__main__':
    # unittest.main()
    # 装载测试用例
    test_cases = unittest.TestLoader().loadTestsFromTestCase(TestStringMethods)
    # 使用测试套件并打包测试用例
    test_suit = unittest.TestSuite()
    test_suit.addTests(test_cases)
    # 运行测试套件，并返回测试结果
    test_result = unittest.TextTestRunner(verbosity=2).run(test_suit)
    #生成测试报告
    print("testsRun:%s" % test_result.testsRun)
    print("failures:%s" % len(test_result.failures))
    print("errors:%s" % len(test_result.errors))
    print("skipped:%s" % len(test_result.skipped))

The output generated after running is:

➜  src git:(master) ✗ python basic_demo.py
test_isupper (__main__.TestStringMethods) ... init by setUp...
FAIL
end by tearDown...
test_split (__main__.TestStringMethods) ... init by setUp...
end by tearDown...
ok
test_upper (__main__.TestStringMethods) ... init by setUp...
end by tearDown...
ok
 
======================================================================
FAIL: test_isupper (__main__.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "basic_demo.py", line 23, in test_isupper
    self.assertTrue('Foo'.isupper())
AssertionError: False is not true
 
----------------------------------------------------------------------
Ran 3 tests in 0.001s
 
FAILED (failures=1)
testsRun:3
failures:1
errors:0
skipped:0

Obviously, the above input results have already counted the test results. These data are important indicators in a test activity. These data can be stored in the database, integrated with the test information management system, and later generate dashboards or statistical reports to form stable and The product test circuit diagram is related to development, so I won’t describe it here.

Combining the specific examples above, we can also find the specific implementation objects corresponding to the theoretical part of the previous section:

test fixture

The initialization work is done by the setUp function, and the destruction work is done by tearDown
test case

Corresponding to the TestCase class, or more detailed corresponding to the test script function inside
test suite

Corresponding to the TestSuite class
test runner

Corresponding to the TextTestRunner class

4 IDE tools

Since the productivity of code development is needed, it is necessary to introduce an IDE tool - Pycharm . It is undeniable that it is currently the most dedicated/professional Python language IDE . There is also better support for Pyunit .

The main support is as follows:

Visual programming development (this is the basic feature of IDE)
Visual display of test results
Export a test report that generates HTML
Visual control use case execution (this is very convenient in the development and debugging stage, it can conveniently control the operation of specified code units)
- Let all commands in a directory execute
- Let all use cases in a single file execute
- Make a single command in a single file execute

4.1 Running and debugging

Pycharm provides flexible running and debugging support for test scripts.

Through pycharm, developers can realize the following functions without writing the main function:

Run all test classes in a file
Run all test scripts of a test class
Run a test script of a test class

Among them, "Run a certain test script of a test class" is more useful, and it is suitable for quickly developing, running and debugging a single script during the development phase.

Instructions:

Move the cursor inside the test function
Press the run shortcut key ctrl+shift+F10 (Eclipse shortcut key scheme)

If you want to debug with breakpoints, use the Debug mode to run and debug with breakpoints for a single function.

Of course, the above functions can also be realized by operating testSuit without borrowing the IDE, but the IDE provides a more flexible and direct choice. These are just some IDE usage skills, so I won't go into details.

4.2 Results Visualization

For the example mentioned above, if you choose to run this program in the IDE, you will see the following effects:

You can see that all runs passed. If one of them is deliberately made to fail, the following results will be displayed:

Python automated testing - automation framework and tools