First, the current trend in software development
It begins with us first briefly introduce some important concepts emerged in recent years in corporate development, in order to introduce the subject of continuous testing. The most important of these concepts is DevOps and two micro-services. Both are currently in software development best practices and methodologies, designed to provide enterprises greater flexibility, improve operational efficiency.
1.1 DevOps
DevOps is a set of practical methodology and culture, organization and advocate breaking the original constraints, functional teams began to embrace and accept the high degree of synergy DevOps advocated, development, testing, operation and maintenance and delivery of integrated thinking. With DevOps agility and heat rising, whether Internet companies or traditional software companies are beginning to embrace agility, practice DevOps. Continuous Integration CI (Continuous integration), continuous delivery CD (Continuous delivery) as a best practice DevOps, and more and more attention.
1.2 micro Services Architecture Microservice Architecture
Micro-service architecture originated in DevOps ideology and practice, it is a software architecture style. Micro-services architecture brings a number of benefits, such as deployability, reliability, availability, and so on. While in principle you can use any architecture practice DevOps, but is becoming a micro-service architecture building continuous deployment (CD) system standard architectural style. Because of the size is very small for each service, which allows a single architecture to achieve continuous service through reconstruction, thus reducing the need for preliminary design of large-scale projects, allowing early release software and continuous delivery. Micro service and DevOps is a natural community, combined together to achieve changes in the software development industry.
Second, automated test Test Automation
With the introduction of agile and micro-service architecture, CI / CD become a standard build and deploy, even in projects not micro-service architecture in. In order to ensure processes and transactions have been defined to run as expected, the test is essential. In response to the frequent changes in modern software products and releases, the traditional manual testing methods in personnel efficiency and there is a serious shortage, so the automated testing has become a key component of modern software development process. Automated testing is the core to get through continuous integration and continuous delivery, there is no effective automated test assurance, continuous integration and continuous delivery to just a body without a soul.
2.1 Test Category
Tested in accordance with different dimensions can be a variety of categories.
- By means of testing whether manual, can be divided into manual testing and automated testing;
- According to the test object can be divided into functional testing, performance testing, load testing.
In this paper, Martin Fowler test are classified according to the classification level of the way.
Martin Fowler describes the test pyramid is divided into units, service and UI three levels. While there was a specific description of this vary (some people are defined as three units, interfaces, integration testing; it was the whole pyramid is divided into 4-5 levels), but bottom-up pyramid structure is recognized and follow.
1) Test Unit
For the test unit tests the code unit (typically a class / method), the value of testing unit is able to provide the fastest feedback, to the logic unit can be verified during development. Good unit tests can help improve both the design team grasp the premise of the TDD, unit testing can assist reconstruction, to help improve the code cleanliness.
2) Interface (Services / API) test
Interface test is a test carried out for the service interface, the main interface functions to achieve internal test is complete. Such internal logic is normal, whether the exception processing correctly. The main value lies in the interface definition interface testing is relatively stable, unlike an interface or the underlying code change frequently, it is easier to write interface testing, maintenance costs embodiment use is also relatively low. In the interface level, ready to test the relatively high cost.
3) Integration (UI) test
Integration test to verify the correctness from the perspective of the user of the product features, the end to end flow measurement, and scene data and adding the user to verify whether the health of the whole process smoothly. The highest business value of integration testing, it is a complete verification process, but because of the need to verify the full process, the deployment environment, ready to use cases and implementation aspects such as high cost, it is not easy to implement.
2.2 micro Services Architecture as a challenge to bring the test
After the micro-service architecture to solve the application size, application development and other issues of scale also brought some new issues, more prominent increase in the number of micro-services have to call the complex relationship between services and so on. Complex dependencies can cause even experienced project developer can not all of a sudden came out with all relationships.
Micro-services and applications compared to traditional monomer, there will be some not the same place on the test strategy. In simple terms, the micro-service architecture, the test becomes more level, need to test services and applications will become more. All the tests manually is inefficient, unable to keep the Internet fast iterative requirements. At this point it is necessary to introduce automated testing to reduce the pressure on the test team, and improve test efficiency and test quality.
2.3 Automated Test
Speaking of automated testing, functional testing personnel might think it is very high-end complex.
First look at how the general functionality tests: Example designed and written documents describing the test steps and expected results; tester as described in Test Case according to steps, and then determining whether the actual result is consistent with expectations. If agreed, the test passes; if not match, the test fails.
Automated test and functional test needs to be done is the same. Stratification theory and method combines automated testing, there are three levels of automation: Automated unit testing, test automation interfaces and UI test automation. Of course, different levels of automation concern is not the same. Therefore, the nature of the test from the behavioral point of view, functional testing and automated unit testing, automated testing interfaces and UI automated test and no difference. The only difference is that performed by a person, executed by a code or tools.
2.4 automated test stratified
1) Automatic Test Unit
Unit test automation, software refers to the smallest unit can be checked and verified test, test class or method calls and services, according to the parameters of the class or method, the corresponding incoming data, returns the result to obtain a final result returned by the assertion whether in line with expectations. If they are equal, the test passes; if not equal, the test fails.
Therefore, the test unit is implemented with the concerned logic of the code. Is the most basic test unit testing, the test unit is the smallest, the object is a function object that may contain input and output for the code logic function or a function of an internal function and does not contain the business logic.
Such tests are generally done by researchers, we need to use a unit testing framework, such as the java Junit, TestNG, python unittest the like.
2) Interface test automation
Interface automated testing, verification calls between the main module and return data exchange between different systems, services. Interface test automation testing in general business logic layer. According to interface documentation is RESTful or RPC? Interface call to be tested, the corresponding configuration data request, return value is success or failure. No matter what input parameters, we will get a result, the final result is equal to the assertion return the expected results. If they are equal, the test passes; if not equal, the test fails.
Therefore, the interface testing are concerned about data. As long as the data is correct, the feature is to make most of the remaining nothing more than to show how these data on the page.
Common interface test tools postman, jmeter, loadrunner and so on.
3) Integration (UI) automated testing
UI layer is the user's entry using the product, all the features available to users through this layer, most of the current test work has focused on this level, this test closer to the user's behavior, simulate a user clicks a button in the input box enter some commands. Sometimes the user may see the login was successful, but the UI Automation does not know it just clicks have no effect. So to find "evidence", such as the login is successful top right corner of the page will display "Welcome, xxx", which is a successful login strong "evidence." When UI Automation successful login, go get this data assert, assert if they are equal, the test passes; if not equal, the test fails.
Therefore, the UI automation point of interest to a user operation type, and the various components is available on the UI. Common test tools UFT, Robot Framework, Selenium, Appium and so on.
4) stratified accounting best practices
Each has its own automated testing and focus on the advantages and disadvantages, divided equally impossible in practice, so we need to develop a reasonable testing strategy and organize its distribution, including how much each part of the test input, test cases the proportion is how much and so on.
Test pyramid also information of another dimension, as shown in FIG.
- Higher up, closer to QA, business / end-user, the more down, closer to development;
- Higher up, test execution slower, more down, the faster test execution;
- Higher up, the higher the cost of test (the more time-consuming, the more blurred the failure of information, the more difficult to track), more down, the lower the cost of test.
According to the test pyramid model and input / output ratio, we learned that the higher the rate of return down, so you should use a lot of unit testing and comprehensive interface testing to cover the basic logic and functional products, the use of a small amount of integration (UI ) test for functional verification of the front-end interface.
Said the industry's best practices to see Google, Google's share of investment in automated tiering are: unit testing (Unit): accounting for 70%; Interface Test (Service): accounting for 20%; integration testing (UI): the proportion of 10 %.
Third, automated testing best practices
Most teams at this stage of the company, is more in line with actual test mode spindle models. The new project, the reason may be due to time limits or developer of habit, did not start the unit test preparation well developed; and some leftover from the old project, that might otherwise not much unit testing.
In the above case, the general practice is to first focus on the test of the intermediate layer, the following two reasons:
- First, the input-output intermediate layer is relatively high, can achieve a higher rate of automation;
- Second, it can help strengthen collaboration among developers with testers to improve test quality. This layer with testers need to develop a common definition, because developers know the details of the internal implementation, testing master business scenarios.
3.1 fusiform transition to Pyramid
When the project for a period of time, the proportion of layers of testing necessary to transition the ideal type of pyramid, then the need to focus on the following three aspects:
- Development and testing of each transfer capability;
- Full attention to the quality of product design with the code;
- Let use cases gradually sinking, and finally a gradual transition to the ideal type.
3.2 Test quality assessment
About metric, do not use single indicator to evaluate and test the quality of products, such as use cases through rate, code coverage and so can not independently assess the quality of products.
When evaluating the quality of the test should be concerned about the following aspects:
- The first embodiment is the ratio, i.e. the ratio of the use case is how much of each layer.
- The second is the test coverage.
- The third test is the total running time, because after after optimization, total running time must be less and less.
- The fourth is the code quality indicators, reflecting the quality and cleanliness of the code.
Fourth, the challenge facing the automated testing
The introduction of automated testing can bring many benefits to the team, of course, test automation has its own drawbacks and challenges. The biggest challenge is to change, because change will lead to a test case fails, so the need for an automated script continues to debug. How to control costs, reduce costs is a challenge to automated testing tools and staff capabilities.
Another point worth noting is that automated testing can not replace manual testing, certain manual exploratory testing is essential. We have been unremitting efforts and exploration, this paper is the first to test automation best practices series of articles, focusing on the current situation and test automation pyramid model, the next series of articles will continue to introduce our automated testing practices, including the core function of automated testing platform, methods and tools of continuous testing.
Author: Wei Jianjun
Source: CreditEase Institute of Technology