Simple sharing of automated testing framework ideas

Table of contents

Foreword:

Layering of test frameworks

Another simple example

Foreword:

An automated testing framework is a structured approach to organizing and managing the process and resources for automated testing. It provides a set of specifications and tools to help testing teams write, execute and maintain automated test scripts more efficiently.

Layering of test frameworks

What does a complete testing framework look like?

For example, if we write appium test cases, if we just do Demo, we will directly use WebDriver API to write. At this time, a use case looks like this:

import os
from time import sleep
from appium import webdriver

if __name__ == '__main__':
    # Returns abs path relative to this file and not cwd
    PATH = lambda p: os.path.abspath(
        os.path.join(os.path.dirname(__file__), p)
    )

    # init driver, open session
    desired_caps = {}
    desired_caps['platformName'] = 'Android'
    desired_caps['platformVersion'] = '4.2'
    desired_caps['deviceName'] = 'Android Emulator'
    desired_caps['app'] = PATH(
        '../../../sample-code/apps/ApiDemos/bin/ApiDemos-debug.apk'
    )

    driver = webdriver.Remote('http://localhost:4723/wd/hub', desired_caps)

    # execute some action
    els = driver.find_elements_by_android_uiautomator("new UiSelector().clickable(true)")

    # check if result is as expected
    if len(els) != 12:
        print "Case Failed. There should be 12 elements but only {} elements exist".format(len(els))
    else:
        print "Case Passed."
     # end the session
    driver.quit()

Then the number of use cases is relatively large, it is too troublesome to start appium in each case, and the test results are not good enough, so we start to use some unit test frameworks, using their setUp, tearDown or test reports. At this point the use case will look like this:

import os
from time import sleep

import unittest

from appium import webdriver

# Returns abs path relative to this file and not cwd
PATH = lambda p: os.path.abspath(
    os.path.join(os.path.dirname(__file__), p)
)

class SimpleAndroidTests(unittest.TestCase):
    def setUp(self):
        desired_caps = {}
        desired_caps['platformName'] = 'Android'
        desired_caps['platformVersion'] = '4.2'
        desired_caps['deviceName'] = 'Android Emulator'
        desired_caps['app'] = PATH(
            '../../../sample-code/apps/ApiDemos/bin/ApiDemos-debug.apk'
        )

        self.driver = webdriver.Remote('http://localhost:4723/wd/hub', desired_caps)

    def tearDown(self):
        # end the session
        self.driver.quit()

    def test_check_clickable_element_count(self):

        els = self.driver.find_elements_by_android_uiautomator("new UiSelector().clickable(true)")
        self.assertEqual(12, len(els))

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(SimpleAndroidTests)
    unittest.TextTestRunner(verbosity=2).run(suite)

Later, we will write more use cases, and the few people who do automated testing alone can't hold it. We need to let some people who are not so strong in technology do it. In order to make them easy to use without messing up our framework, we will repackage the use cases and make them into tables or BDDs, which can be written and read more easily by people who do not know how to code. At this point the use case will look like this (Cucumber):

Feature: Simple android test
  Do some sample operations with android application

  Scenario: Check clickable element count
    When I opened the application
    Then 12 clickable buttons should occur

These three steps actually correspond to the three levels of the test framework: the tool layer (such as appium), the core layer (such as the unit test framework), and the adaptation layer (such as the BDD framework)

tool layer

The tool layer is mainly responsible for the action triggering of the test execution on the corresponding side. (Original text in the book)

First of all, if we want to control some tested programs through programs, we must have corresponding tools. These tools make possible controls that would otherwise be difficult or even impossible to achieve, while also making them easier to achieve. For example UIAutomation for iOS. If it does not exist, we must add some agents (such as MonkeyTalk) to the application, or directly add test code (such as KIF) to the application, or even make a set of testing tools that can control the program through the interface. These tools allow us to save a lot of time and make it easier to separate the testing part from the functional part.

There are many frameworks for the tool layer, such as Appium, robotium, espresso, etc. for mobile testing, and selenium for web testing.

core layer

The core layer is responsible for the driving of test execution and result monitoring and feedback. (Original text in the book)

Automated test programs actually have some things in common. For example, they all need rich assertion function support, otherwise they can only write various If..else.., and then the verification and execution steps are coupled together. This is what the framework at the core layer does. It extracts the most common features of programs such as automated test programs (for example, special exceptions need to correspond to fail, various flexible and convenient assertions need to be created, and preconditions need to be created. Special functions), and then encapsulate them in a relatively fixed way of writing.

For example, most use cases will have preconditions (for example, web use cases will require the browser to be open), and will be highly repetitive, and have the characteristics of blocks (if the browser is not opened, the subsequent steps will not be executed). Then the core layer framework will provide a special method (the general method is called setUp), let this method be executed before executing the content of the use case to create a suitable precondition, and automatically put the entire use case when this method fails or fails Mark as fail or block.

There are also many frameworks at the core layer, such as various unit testing frameworks (JUnit, OCUnit, etc.), Testng, etc. Generally speaking, after the framework used by the tool layer is selected, the language used will be determined, so the tool layer is generally determined first, and then the framework used by the core layer is selected. If the languages are inconsistent, appropriate secondary development is required to facilitate calling. Of course, tool layer frameworks like appium/selenium that support multiple languages can choose core layer frameworks more flexibly, which is also an important reason for their popularity.

adaptation layer

The adaptation layer is responsible for reusable test method package adaptation. (Original text in the book)

When the use cases reach a certain order of magnitude (tens or hundreds), if the repeated parts are not extracted well or written in a better form, a certain amount of redundant code will appear. At this time, the role of the adaptation layer is to better encapsulate these repeated methods, so that those who write use cases can write/read use cases more quickly.

Most of the data-driven, keyword-driven, and behavior-driven (BDD) we usually hear belong to this layer. Because no matter what driver is used, it has nothing to do with the specific test program/test domain. You would not say that BDD can only be used to test Android applications, nor that data drivers can only be written in Java. These xx drivers are mainly an idea, a use case writing idea that is easy to use in practice, practical, and has many tools to choose from. Of course, due to the different tools used, the language used will also have certain restrictions, but the ideas are the same.

The adaptation layer also has various frameworks, such as BDD's Cucumber, keyword-driven robot framework (it also has a core layer and a tool layer, which is a complete and practical test framework, and its tool layer is in the form of an extended library Provided, very easy to expand to adapt to different software fields, it is worth learning). Since the writing method of the adaptation layer probably does not belong to a specific programming language (such as keyword-driven using the form of a table, which is separated from the language), most of the frameworks of the adaptation layer will have a corresponding core layer/tool layer The framework provides integrated support.

a simple example

I believe many people have used robot framework. It is a complete testing framework with these three layers:

Adaptation layer : Use cases are written in tsv format, which is mainly keyword-driven.
Core layer : Provides setUp and tearDown methods, and has suitable and sufficient assertion functions and exception capture functions.
Tool layer : There are many libraries to choose from, which can be combined with different tools to test software in various fields.

Another simple example

Another example is an implementation of a keyword-driven framework:

Adaptation layer : use cases presented in tabular form, and converters that convert use cases into executable code:

sheet:

action	params
openBrowser	browserName="Chrome"

Converted code:

class TestCase(ActionBase, unittest.TestCase):

    # a test case
    def open_browser(self):
        # step of test case
        self.action_call("openBrowser", {"browserName":"Chrome"})

Then the implementation of this action_call calls the corresponding action method to perform the actual action:

class ActionBase:
    ...
    def action_call(action_name, params):

        # get action function in this class
        action_fun = getattr(self, action_name)

        # execute action
        action_fun(**kwargs)

    ...
    def openBrowser(browserName=None):
        if browserName == 'Chrome':
            self.driver = webdriver.Chrome()
        ...

Core layer : The converted code uses the unittest framework to manage use case execution

Tool layer : the selenium framework is used when the action is actually executed

As someone who has been here, I also hope that everyone will avoid some detours

Here I will share with you some necessities of the way forward in automated testing, hoping to help you.

(software testing related materials, automated testing related materials, technical questions and answers, etc.)

I believe it can make you better progress!

Click on the small card below