Based on open source frameworks and technologies such as Robot Framework, Jenkins, Appium, Selenium, Requests, AutoIt, etc., we have successfully built a general automated test continuous integration management platform (hereinafter referred to as the "platform"), which has significantly improved the test quality and the execution efficiency of test cases.
01. Design goals
The platform is universal and supports different types of automated testing
The platform framework is lightweight, open and flexible, and has a powerful and rich test library, which fully supports PC desktop applications (such as core counters and online trading systems), Web applications (such as mall online halls), APP applications (such as mobile securities), interfaces ( Such as counter peripheral interface), database (such as Oracle) and other automated tests, which facilitate subsequent continuous test development and centralized and unified management.
The platform supports the whole process of automated testing
The platform display interface is unified, clear, and concise, covering the processes of test case design, test script development, test execution, and test report generation. Through the unified management of the testing process, the threshold for testers to use the platform is lowered.
The platform has good scalability and easy maintenance
The platform supports collaborative work and distributed test execution, and can be integrated with version management tools, software defect management systems, and enterprise email systems. At the same time, automated test scripts can be flexibly adjusted and code reused during the iterative process of the project.
02. Architecture design
The platform adopts the idea of componentization and layered architecture, which can be divided into 4 layers. Each layer implements specific functions and provides interfaces for other layers to call:
The bottom layer is the system under test, which mainly defines the form of the object under test. The platform supports APP, Web and PC application testing, as well as testing of interfaces and databases.
Up is the test library. In addition to providing basic test libraries such as processes, strings, collections, and screenshot processing, the platform also provides extended test libraries for special tests such as Web, APP, and interfaces to support different systems under test to select corresponding test libraries.
In the middle is the test framework, whose functions include use case management, test data management, test operation, test report, etc.
The top layer is the continuous construction layer, which is responsible for the control and scheduling of tasks, machines, queuing, etc., and automatically completes the entire test operation process.
03. Platform realization
In the selection of automation framework, we choose Robot Framework (hereinafter referred to as "RF") as the core underlying framework of the automation test platform, Jenkins as the continuous integration and task scheduling platform, and AppiumLibrary, Selenium2Library, RequestsLibrary, AutoItLibrary, etc. as the extended test library , to meet different testing requirements.
(1) RF framework
RF is a feature-rich and extensible open source automated testing framework, mainly used for regression testing with many rounds. The framework forms test logic through the combination of keywords, and finally interacts with the system under test through various underlying test libraries to complete the test work.
The RF framework provides standard underlying keyword APIs for APP, Web, interface, database testing, etc. Testers only need to develop custom scripts based on these keywords to meet their own testing requirements.
（2）AppiumLibrary
Appium is an open source, multi-platform and multi-language testing framework. Compared with other frameworks, it supports native, Web and hybrid application automation testing on iOS and Android platforms. It can reuse code between platforms, write test scripts and run tests. There is no need to recompile the source code, there are not many requirements for the programming language in script writing and implementation, and the test is lighter and more flexible.
AppiumLibrary is a third-party extended application testing library based on Appium in the RF framework, which can support the automated testing and development of mobile securities APPs for brokerages.
（3）Selenium2Library
Selenium is an automated testing tool suite specially designed to implement multi-platform Web application testing, with features such as scalability, cross-platform, and multi-browser support. Selenium 2 integrates WebDriver. WebDriver directly controls browsers through native browser support or browser extensions. Testers can choose different WebDrivers to perform tests according to different platforms and browsers.
Selenium2Library is a web test library based on Selenium2 for RF framework, which can support automated test development of web applications such as brokerage malls and Internet halls.
（4）RequestsLibrary
Interface testing is generally used for interactive development between multiple systems, or testing of application system development with multiple subsystems. Requests is written in Python language, based on urllib, and adopts HTTP library of Apache2 Licensed open source protocol, which can meet the requirements of HTTP interface testing. RequestsLibrary is an API test library under the RF framework, which can support automated test development of brokerage IFS or other custom development interfaces.
(5) AutoIt desktop application test
AutoIt is a software for automating Windows GUI, which can simulate keyboard keys, mouse movements, and window or control operations to achieve automated tasks. AutoItLibrary is a Win32 desktop application test library based on AutoIt for the RF framework, which can support the automated test development of desktop application systems such as brokerage core counters and online transactions.
(6) Jenkins continuous integration
Jenkins is an open source continuous integration tool that supports multiple SCM configuration management tools such as svn and Git. It can set a scheduled build (such as hourly, every interval, or what time every night) or trigger a build according to project requirements. (such as source code submission trigger), supports Shell script, Windows batch processing, Ant construction and other construction methods, so as to meet the requirements of running on different operating systems.
Jenkins supports distributed construction (similar to Selenium Grid), which can realize multi-host multi-task parallel execution. Jenkins also provides rich plug-in support. After installing the Robot Framework plug-in plug-in, RF automation test cases can be executed.
04. Innovation points of the platform
1. The platform structure is flexible, adaptable to a wide range of scenarios, and meets various types of automated testing
The platform is based on a lightweight RF open source framework and has good scalability. By integrating different open source extended test libraries, it can meet the tests of different types, different platforms and applications:
RF+AppiumLibrary can realize APP automated testing, and can reuse code between iOS and Android, only need to write a unified test script to run on different platforms, this method will greatly improve the use of automated test scripts efficiency, saving time in writing and debugging test scripts;
RF+Selenium2Library can realize the automated testing of Web applications under multiple browsers such as Chrome, IE, and Firefox;
RF+AutoItLibrary can realize automatic testing of general Win32 desktop applications;
RF+RequestsLibrary can realize interface automation testing, etc.
Therefore, the platform has strong flexibility, compatibility and ease of use, especially suitable for automatic testing of core counters, online transactions, mobile securities and other interfaces that do not change much, have a long software life cycle, and often launch new versions of products.
2. The platform realizes end-to-end automated test management, and one-stop graphic display information
The platform supports the whole process of use case management, script development and debugging, task distributed scheduling and execution, and test report presentation
3. Based on data and keyword-driven automated testing, flexibly respond to changes in the system under test
The platform overcomes the shortcomings of the high coupling of scripts in the recording and playback mode of traditional automation frameworks, and has high robustness, high reusability and good maintainability:
Supports data-driven mode, separates test data and scripts, and changes test data without modifying scripts, which improves the reusability of automated test scripts.
Unified management of interface elements is realized, interface elements and test scripts are separated, and application interface changes only need to modify invalid interface elements, without a lot of script modification, which improves the maintainability of scripts.
The business process test is controlled based on keyword scripts that can be flexibly combined, and the function extension of the test framework can be realized by adding keywords.
The script structure adopts the layered design idea, and the automated test script is divided into a use case layer, a process layer, a page element layer and a variable collection layer, which facilitates the realization of high cohesion and low coupling test script design.
05. Implementation effect of the platform
1. Improve the quality of test case design and the coverage of business scenarios
Since the implementation of Dongguan Securities Automation Test Project, the granularity and coverage of use cases have been greatly improved. The test cases of mobile APP transactions, interfaces, Web (net hall/mall/youcailai), core counters, and online transactions have reached 1645, 153, 76, 569, and 1168, with an average increase of nearly 8 times in the size of use cases, basically covering all transaction businesses. The average automation rate is over 65%, which greatly improves the quality confidence of the product development team.
2. Improve the efficiency of test case execution and speed up product launch
Automated testing improves the efficiency and accuracy of test case execution and reduces the failure rate of production application systems. In addition, continuous integration intervenes as early as possible, reducing the possibility of delayed version releases. Take Dongguan Securities’ mobile app transaction automation test as an example. The iteration cycle of the previous version was 2-3 months, but it was shortened to 1 month after the automation test was carried out; the previous round of 15-man-day manual test was changed to less than 2 people after automated execution sky. Calculated on an average monthly iteration, nearly 150 man-days of testing resources can be saved throughout the year. If it is connected to an independent test counter (supporting 7*24 execution) and multiple devices are added to execute in parallel, the efficiency of automated testing can be greatly improved; the project has invested nearly 20 rounds of iterative testing, and 17 defects have been found.
06. Summary
Based on the open source RF framework, Dongguan Securities has established a general automated testing system from 0 to 1, and implemented it in projects such as mobile APP, core counters, online transactions, interfaces, malls, and online halls, and combined with Jenkins to realize continuous integration and distribution of automated testing The model construction effectively supports the agile and fast iterative process.
Due to space limitations, this article does not discuss in detail the automated test development methods such as APP, Web, desktop application, interface, etc., but provides an idea of establishing a general automated test continuous integration management platform, hoping to inspire and learn from other securities companies significance.
Finally, I would like to thank everyone who has read my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, you can take it away if you need it:
Software testing interview applet
The software test question bank maxed out by millions of people! ! ! Who is who knows! ! ! The most comprehensive quiz mini program on the whole network, you can use your mobile phone to do the quizzes, on the subway or on the bus, roll it up!
The following interview question sections are covered:
1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux
6. web, app, interface automation, 7. performance testing, 8. programming basics, 9. hr interview questions, 10. open test questions, 11. security testing, 12. computer basics
These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey. I hope it can help you too!