Preface
Automated testing has gradually become the "trend" of testers due to its advantages such as cost saving, increased efficiency, and reduced manual intervention. Practitioners have increasingly clearly understood that the realization of an automation framework is one of the key factors for the success of software automation projects. This article will explain three aspects of what is a real automated testing framework, how automated scripts work, and how automated testing frameworks will provide you with a competitive advantage during the testing process. Through this article, you can step into automated testing. The field of the frame.
Advantages of automated testing
The popularity and many advantages of automated testing do not mean that it is a panacea. Customized projects, short cycle times, and ease of testing are not suitable for automated testing. But it is undeniable that if you carefully plan and execute the automated testing framework, it will indeed bring the following benefits to software development and testing companies:
Minimum time-maximum benefit
Building any feasible automated testing framework and automated scripts can minimize the time it takes to write and run tests, so that the maximum output can be obtained in a short time. In addition, with an excellent automated testing framework, manpower can be freed from common problems such as synchronization, error management, local configuration, report generation, and interpretation.
Reusable and readable automation code
When you use the code mentioned in the existing component library, you can rest assured that it will still be readable and reusable in the future, and all related tasks (such as reporting, synchronization, and troubleshooting) will change To be more accessible.
Resource optimization
Some companies do not benefit from automation implementation as envisioned, because the benefits from automated testing depend on the flexibility of adopting it. If the automation system is flexible and compatible with different teams working on each component, it can provide huge benefits in terms of resource optimization and knowledge sharing.
Definition of automated testing framework
In many other industries, "automation" refers to the automatic processing of processes through intelligent algorithms, and the process runs with little or no manual intervention. In the software industry, automated testing refers to various tests of software applications through automated tools (licensed or open source versions are available). From a technical point of view, an automated testing framework is a set of customized interactive components that can be used to execute scripted tests and comprehensively report test results.
To successfully build an automated testing framework, you need to take into account the recommendations of software quality assurance experts. They can help observe and help control the entire testing process and improve the accuracy of test results. A well-designed automated testing framework allows testers to perform automated testing in a practical and simplified way.
Types of automated testing frameworks
Tool-centric framework
Whether it is commercial automation tools or open source automation tools, there are system infrastructures that help generate reports, test suites, and distributed test execution in a test environment. For example, the Selenium automation framework has the main component WebDriver-a web browser-based plug-in, which is used to control and manipulate the DOM model of the application in the web browser. The Selenium test automation framework also has practical coding libraries and record playback tools.
Another important framework dedicated to tools is Serenity, which is an accelerator built around the Selenium Web driver. In this case, in order to speed up the test automation implementation process, the community combines specific components into a common entity.
It is difficult to make clear judgments when using tool-specific frameworks such as TestComplete and Ranorex HP QTP, because they are all pre-built with a deployed infrastructure with action simulators, reports, and scripting IDEs.
Application-centric framework
This type of framework is customized to automate a specific application project. The application-centric framework supports the needs of a specific target application to achieve test automation, driven by components built from open source libraries. This framework can create a friendly test environment around SUT to run some basic functions, including deploying developed applications, running applications, test case execution, direct test result reports, and easy-to-code wrapper controls. The application-centric framework will also have a component to support test runs across various cloud environments on different operating systems and browsers.
Keyword Driven Framework
Keyword-driven framework can mainly attract novice developers and testers who lack coding experience. It can be a tool-centric framework or a project-centric framework. The keyword-driven framework allows employees who are not skilled in coding to participate in writing and understanding automated scripts. The keyword set used for coding (such as Login, NavigateToPage, Click, TypeText) is installed in the code library as a keyword repository, and the spreadsheet for writing scripts based on the provided keyword references will be passed to the keyword interpreter for testing .
The main components of an ideal automated testing framework
If you want to implement a powerful and high-performance test automation framework (whether open source or commercial), you must take into account certain components that constitute its core. Some frameworks may include all components, and some frameworks only include a few of them, so it is not necessary to involve all the components mentioned below in each framework.
1. Test basic components
a) Unit testing
Unit testing basic components can be used as a basic part of any automated testing framework. It can be used in the following situations:
● Use specific forms such as @test or [test] to define the test method being used
● Execute assertions that affect the final result of automated testing
● Run simple and clear tests
Whether running tests from the command line, IDE, special tools or CI (Continuous Integration) systems, in order to ensure that unit tests are run in a simple way, the unit test library provides a test runner.
The unit test library supports almost all programming languages, such as the following: JUnit and TestNG for Java, NUnit and MSTest for .NET, and unittest for Python (formerly PyUnit).
b) Integration and end-to-end testing
While performing integration and end-to-end test automation, it is also very suitable to practice the functions provided by the existing test library. API-level testing driven by the application UI needs to have components that eliminate unnecessary coding burdens, so that the interaction with the application under test becomes easier. Therefore, the tester will not be troubled by the coding work of connecting to the application, sending the request, and receiving the result response.
Several important test components of this type are: Selenium (available for the main language), protractor (specific to JavaScript), Karate DSL (Java-specific API-level integration testing).
c) Behavior Driven Development (BDD)
Components dedicated to BDD target behavioral specifications and create executable specifications in the form of executable code. In this part, testers can convert different characteristics and scenarios of predictive behavior into code. Although unlike other testing tools that directly interact with the application under test, it can be used as a support for the BDD process to create active documents consistent with the scope and intent of automated testing. Typical examples of BDD components are: Cucumber (supporting major languages), Jasmine (JavaScript), SpecFlow (for .NET).
2. Test data management
In the process of software test automation and test creation, the biggest difficulty is how to use the test data management system. As the number of automated tests increases, there is always the problem of ensuring that certain test data required to perform a particular test is available or can be created when the test is performed. At present, there is no foolproof solution to this situation, which requires the use of reliable test data management methods to make automation work successfully.
Therefore, the adopted automated testing framework needs to be fully configured to provide basic remedial measures to enter or create and clear the test data to be executed. The solution to this problem is to use appropriate simulation tools to make the data more simplified, clearer, and easier to understand.
3. Mock, stubs and virtual assets
In the process of practicing and exploring automated testing, the following situations may occur:
● Want to isolate the module from connected components that have experience in unit testing
● Deal with the tedious and critical dependencies that are common in modern application integration or end-to-end testing
In these cases, it is necessary to create mocks, stubs, and virtual assets to reflect the behavior of connected components. You may also find that handling mocks and stubs is a large-scale and large-scale task. In any case, testers will realize that it is very important to choose useful virtualization tools during the development of automated testing frameworks.
General mechanism for implementing patterns
In addition to the automation framework components mentioned above, there are some useful mechanisms to help create, use and maintain automated tests, such as:
Wrapper methods: When using the Selenium WebDriver component, creating a custom wrapper will make it easier to handle errors. Once you have created a custom wrapper for Selenium API calls, you can better handle timeouts, exception handling, and failure reporting. After that, the person who created the automated test can reuse it, so that you can avoid the concerns of complex processes and focus on valuable tests.
Abstraction methods: Abstraction mechanisms mean improved readability and concealment of redundant details. For example, the purpose of using page objects when creating a Selenium WebDriver test is to expose user input operations on the web page, including entering credentials or clicking somewhere on the page. The goal is to achieve advanced levels by going beyond or bypassing the need to explore specific elements on the page testing method. This method is suitable for many similar applications and automated tests.
Test result report
When choosing a library or mechanism for reporting test results to an automation framework, you need to take into account the target audience viewing the report. In this regard, the following precautions should be followed:
● The reports generated by unit testing frameworks such as Junit and TestNG are mainly for receiving systems such as CI (Continuous Integration) servers, which will eventually interpret them and present them in an XML format that can be used by other software.
● When seeking to use a language that most people can understand as a reporting tool, you may want to consider using commercial tools compatible with unit testing frameworks, such as UFT-Pro for Junit, NUnit, and TestNG.
● Another option is to utilize a third-party library such as ExtentReports, which creates test result reports in an easy-to-understand format, including visual interpretation through pie charts, graphs, or images.
Auxiliary platform
In addition to the automated testing framework itself, some auxiliary platforms related to continuous integration will also be of great help to automated testing. For example, CI can perform deliverable deployment and test scheduling; the source code platform can manage the code of products and test scripts; the dependency manager is generally a dependency management tool for a certain language class library, such as the general java type developed by nexus management company employees ; ZenTao ZTF includes features such as continuous integration, unit testing and version library management.
CI platform
In order to conduct application testing in a faster and more coherent manner, a continuous integration platform can help you regularly build software and run various tests for new versions. When developing and deploying new features and updating existing features, this method allows developers and testers to get regular feedback and rapid response to the quality of the application. Some prominent continuous integration platforms currently include TeamCity, CircleCI, Jenkins, etc.
Source code management
Like manual testing, automated testing also involves writing and storing source code versions. Each development company has a selected source code and version control system to save and protect the source code. Automated testing requires a complete source code management system, which is very convenient when dealing with production code. Typical source code management examples are Git and Subversion.
Create a dependency manager
The main purpose of the dependency manager is to assist in the collection and management of existing dependencies and libraries that have been applied to the functions of the automation software solution. Certain tools (such as Maven and Gradle) also act as dependency managers and help build tools. The build tool is designed to help you develop automated software and run tests from source code and supporting libraries. Tools in this area include Ant, npm and NuGet.
Zen Tao ZTF
ZTF supports seamless integration with ZenTao, which can bind ZenTao use cases and automated test scripts one by one. The execution results can be submitted to ZenTao as test results, and scripts that fail to execute can also automatically create bugs. The ZTF automated testing framework has realized continuous integration with Jenkins. After the user initiates the task, the test script is automatically executed through ZTF, and the result of the unit test is passed back to ZenTao. The two cooperated to open up the continuous integration loop.
Framework construction and implementation process
The following are several ways to plan and implement automated testing solutions:
● Discuss the actual applicability of automation from the customer's point of view, check whether its appearance is good from all angles, and test on underused technologies. In contrast, if the effort of automated development greatly exceeds the expected advantages, then automated testing is unnecessary.
● Pay close attention to the technology of the system under test in order to find the most suitable automated testing tool that can perfectly simulate user behavior.
● It is recommended to adopt a phase-based implementation method, where each phase has the priority of delivering automated test scripts, and at the same time, adding framework functions to achieve the expected script execution.
● Before starting software automation testing, in order to ensure the correct implementation of automated decisions, you must first estimate the return on investment after implementation, proof of concept, time to run manual regression or smoke testing, and the number of operating cycles for each version.
in conclusion
In today's fast-paced and brutal software development ecosystem, automated testing frameworks play an indispensable role in maintaining the speed, efficiency and clarity of the software testing cycle. With the introduction of artificial intelligence into software testing, organizations considering the use of automated testing frameworks must delve into the design of the final framework before entering this field. A good framework design and a strategy of using components will lay the foundation for the final automated testing framework.
The best way to build a mature, sophisticated, and flexible automated testing framework is to start small, conduct frequent tests and audits, and gradually build an extended version to a higher level. It is convenient to prepare a large number of automated tests from the early stage. You can see the working framework more quickly and avoid conflicts or damages in the automated testing phase.
The guidelines set out above are designed to help software testers and companies so that everyone can learn from the successful implementation of automated testing projects.
Finally: benefits
In the technology industry, you must improve your technical skills and enrich your practical experience in automation projects. This will be very helpful for your career planning in the next few years and the depth of your testing technology.
In the interview season of the Golden 9th and the Silver 10th, the job-hopping season, organizing interview questions has become my habit for many years! The following is my collection and sorting in recent years, the whole is organized around [software testing], the main content includes: python automation test exclusive video, Python automation details, a full set of interview questions and other knowledge content.
For software testing friends, it should be the most comprehensive and complete interview preparation warehouse. In order to better organize each module, I also refer to many high-quality blog posts and projects on the Internet, and strive not to miss every knowledge point. Friends relied on these contents to review and got offers from big factories such as BATJ. This warehouse has also helped many learners of software testing, and I hope it can help you too!
May you and I meet and you will find something! Welcome to follow the WeChat public account: [Sad Spicy Article] Receive a 216-page software test engineer interview book for free. And the corresponding video learning tutorials are free to share!
