Talk about python automated testing (3)-automation framework and tools

Python automated testing(3)

Automation framework and tools

1 Overview

The methodology for testing procedures is based on the points mentioned in the previous article:

• Functional testing is not recommended to be automated
• The most cost-effective interface test
• Interface testing can be automated

The test automation discussed later will also be introduced around interface automation.

The test language selected for this series is the python scripting language. Since its official document has a clearer explanation of the principle, this article will not do some redundant translation work. It is biased towards actual combat, and in order to be biased towards actual combat, it will also be explained in conjunction with IDE tools and project organization.

The reasons are as follows:

1. Scripting language, highly efficient in development and iteration

2. There are a lot of third-party extension libraries, and there are many ready-made tools that I can use

Before officially entering the field of automated testing, we must first establish such values. It is mentioned in the software testing publication issued by Google internal engineers:

"The automated testing of software has a cost, and the cost is not low, basically equivalent to establishing a parallel test development project on the basis of the original functional development project."

In other words, if you have your expectations for automated testing, you must pay the corresponding price and energy. Good things also require great people to spend a lot of time to complete.

This article has been included in the collection: "Python-based Internet Software Testing and Development (Automated Testing)-Complete Collection", welcome to visit:

Python-based Internet software testing and development

2 PyUnit test framework

Using python as an automated programming language, then it is natural to use pyunit as an automated testing framework.

The content of the following part mainly comes from the official documents of pyunit. This article only makes some simple adjustments in translation and structure. This part belongs to the basic principles and concepts of the testing framework. It is necessary to understand before writing the code.

PyUnit, the unit testing framework of python, can be considered as the Python language implementation version of the unit testing framework JUnit under the Java language. Even one of its authors, Kent Beck, is the author of JUnit.

Unittest has to achieve the following goals:

• Support automated testing
• Let all test scripts share the code for opening (setup) and closing (shutdown)
• Test case scripts can be organized by means of collections
• Separate all test scripts from the test report framework

In order to achieve the above goals, unittest supports the following important concepts:

• Test fixture (test fixture) Do some preparations for one or more test cases, for example: connect to a database, create a directory, or start a process

• Test case (test case) A test case is the smallest unit of test behavior, which is tested and checked by comparing some input and output values

• Test suite (test suite) A collection organized by aggregating test cases or a collection of test cases. Can batch execute all test cases in a test suite

• The test runner
  organizes and arranges the components of the test script execution activities. The test executor displays the test results of the test script through some graphical interfaces, text interfaces or returning some special values. Mainly used to generate test reports

3 Basic example

The following example is also from the official document basic_demo.py:

# coding:utf-8
"""
基本的自动化测试脚本 basic_demo.py
"""
__author__ = 'zheng'

import unittest


class TestStringMethods(unittest.TestCase):

    def setUp(self):
        print 'init by setUp...'

    def tearDown(self):
        print 'end by tearDown...'

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())
        self.assertTrue('Foo'.isupper())

    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)


if __name__ == '__main__':
    unittest.main()

Although the official document introduces several ways to organize test case scripts:

1. Independent test function

2. Single use case test class

3. Multi-use case test class

Different forms of writing will have different organization methods. For details, please refer to official documents. After studying the official documents, the author of this article most likes the third method, multi-use case test class, which is the basic example method above. This method has the following characteristics:

• The test class inherits from unittest.TestCase
• One test class can manage multiple test script functions
• The test script function name needs to start with test_
• All test functions in a test class share the setUp and tearDown functions
• Run this program in the console:

➜  src git:(master) ✗ python basic_demo.py
init by setUp...
Fend by tearDown...
init by setUp...
end by tearDown...
.init by setUp...
end by tearDown...
.
======================================================================
FAIL: test_isupper (__main__.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "basic_demo.py", line 24, in test_isupper
    self.assertTrue('Foo'.isupper())
AssertionError: False is not true

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)
➜  src git:(master) ✗

The simplest method used by the main function of the previous basic example is to directly run all test cases and generate a default text report. In fact, you only need to make some simple modifications to the calling function, you can organize these test cases reasonably, and obtain useful data information in order to integrate with the information system to form a better extension.

if __name__ == '__main__':
    # unittest.main()
    # 装载测试用例
    test_cases = unittest.TestLoader().loadTestsFromTestCase(TestStringMethods)
    # 使用测试套件并打包测试用例
    test_suit = unittest.TestSuite()
    test_suit.addTests(test_cases)
    # 运行测试套件，并返回测试结果
    test_result = unittest.TextTestRunner(verbosity=2).run(test_suit)
    #生成测试报告
    print("testsRun:%s" % test_result.testsRun)
    print("failures:%s" % len(test_result.failures))
    print("errors:%s" % len(test_result.errors))
    print("skipped:%s" % len(test_result.skipped))

The output generated after running is:

➜  src git:(master) ✗ python basic_demo.py
test_isupper (__main__.TestStringMethods) ... init by setUp...
FAIL
end by tearDown...
test_split (__main__.TestStringMethods) ... init by setUp...
end by tearDown...
ok
test_upper (__main__.TestStringMethods) ... init by setUp...
end by tearDown...
ok

======================================================================
FAIL: test_isupper (__main__.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "basic_demo.py", line 23, in test_isupper
    self.assertTrue('Foo'.isupper())
AssertionError: False is not true

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)
testsRun:3
failures:1
errors:0
skipped:0

Obviously the above input results have been statistically the test results. These data are important indicators in a test activity. These data can be stored in the database and integrated with the test information management system. Dashboards or statistical reports are generated later to form a stable and Product test circuit diagrams, these are all related to development, so I won’t repeat them here.

Combining the above specific examples, we can also find the specific implementation objects corresponding to the theoretical part of the previous section:

• The test fixture is initialized by the setUp function, and destroyed by tearDown

• Test case (test case) corresponds to the TestCase class, or a more detailed corresponding to the test script function inside

• Test suite (test suite) corresponds to the TestSuite class

• Test runner (test runner) corresponds to the TextTestRunner class

4 IDE tools

Since you need to develop code productivity, then you need to introduce an IDE tool-Pycharm. Undeniably, it is currently the most dedicated/professional Python language IDE. There is also better support for Pyunit.

The main support is as follows:

• Visual programming development (this is the basic feature of IDE)

• Visual display of test results

• Export and generate HTML test report

• Visually control the execution of use cases (this is very convenient in the development and debugging stage, and can easily control the operation of the specified code unit)

  a. Let all commands in a directory execute
  b. Let all use cases in
  a single file execute c. Let a single command execute in a single file

4.1 Operation and debugging

Pycharm provides flexible running and debugging support for test scripts.

With pycharm, developers can achieve the following functions without writing the main function:

• Run all test classes in a file
• Run all test scripts of a test class
• Run a test script of a test class

Among them, "run a certain test script of a test class" is more useful, suitable for rapid development and running debugging of a single script in the development phase.

Instructions:

1. Move the cursor inside the test function

2. Press the run shortcut ctrl+shift+F10 (Eclipse shortcut key scheme)

If you want to debug with a breakpoint, use the Debug mode to run a single function and debug with a breakpoint.

Of course, you can also realize the above functions without borrowing the IDE, and by operating testSuit, but the IDE provides a more flexible and direct choice. These are just some IDE use skills, not to mention them.

4.2 Visualization of results

For the aforementioned example, if you choose to run this program in the IDE, you will see the following effects:

You can see that all have passed. If one of them is deliberately failed, the following result will be displayed:

4.3 Generate test report

Pycharm also provides the export function of the test result report, on a function button on the test result display box.

The export results are as follows:

Of course, if you do not consider the integration with the information system, the subsequent dashboard and test statistics work, just to generate reports, this function is sufficient.

Under normal circumstances, for automated testing and development, the above skills are fully capable of meeting the requirements. The next thing to do is to use various basic computer knowledge to face the ever-increasing business needs and continuously increase test cases. The script is out.

The principles of functional development projects are very simple, but as the volume increases, they will form a scale, and the same is true for test development projects.

5 Project organization

Previously, we introduced the development and debugging tools of test cases. But if you really want to be included in the continuous integration automation system, you obviously cannot rely on the IDE. Instead, it uses the organization and calling methods of the python language, such as: the main function is required as the execution entry, and so on.

For detailed technical implementation details, I will write a corresponding article to introduce when I have the opportunity later.

Through the project organization method out of IDE, there are the following advantages:

All scripts can be executed by event triggering (it can become a part of continuous integration pipeline)
All data can be proposed and customized processing and processing (integrated with test information system, providing data source for quality analysis system)

6 Test platform

Regarding how to automatically generate a test report, this test product, there are now some platforms that can provide interface calling and report display and sharing functions. For details, please refer to: https://blog.csdn.net/weixin_50271247/article/details/109265422

7 Summary

The content of this small part mainly talks about some design ideas and basic usage examples of pyunit, an automated testing framework based on the python language. In fact, the use of tools is very simple, but how to make good use of these tools for software production requires other computer skills. In subsequent articles, the application of this framework will be extended in depth from the engineering and technical aspects. .

Software testing is the easiest subject in IT-related industries to get started~ It does not require the logical thinking of developers, and operations and maintenance personnel are not required to be on call 24 hours a day. What is needed is a careful attitude and a broad understanding of IT-related knowledge. The growth path of each tester from entering the industry to becoming a professional expert can be divided into three stages: software testing, automated testing, and test development engineers.

Here are some information I have compiled. If you don’t want to experience the self-study again, you can’t find the information, no one answers the question, and you feel like giving up after a few days, you can add our software testing exchange group 313782132, which contains various software Test data and technical exchanges.