Table of contents
The meaning of interface testing
Interface test framework selection
Our encapsulated interface testing framework
Key Practices of Interface Testing
Test code specification (for reference only)
Foreword:
The interface automated testing framework based on JUnit can realize the automated testing of the interface, and provides rich assertion and reporting functions. JUnit is a popular Java unit testing framework, which can help developers write repeatable test cases and verify test results.
Layered automated testing
5~10 years ago, the automated testing we came into contact with focused more on the automated testing of the UI layer. Mercury's WinRunner/QTP was a typical representative of commercial automated testing products in that era. Automated tools replace human clicks, and commercial or privatization frameworks are popular.
And layered automated testing advocates that automated testing is required at different stages (levels) of the product. In "The Way of Google Software Testing", 70% of Google's investment is unit testing (small testing), 20% is interface/integration testing (medium testing), and 10% is automated testing of the UI layer (large testing). It is the pyramid model that everyone is familiar with. The higher the level is, the more difficult it is to implement automation, and the lower the income from investment (it should be emphasized that, as the test closest to user operations, automated testing at the UI layer still has its meaning and scenarios. ).
The meaning of interface testing
Interface testing is to verify the interaction between two or more module applications (usually in the form of interfaces). The focus of the test is to check the data exchange, transfer and control management process, including the number of times of processing.
The core strategy of interface testing is: to ensure the correctness and stability of the system as the core, and to use continuous integration as a means to improve test efficiency, improve user experience, and reduce product development costs.
Interface testing should escort the writing of code, enhance the confidence of developers and testers, expose hidden bugs in advance, let developers fix bugs in the first time, and make business testers more comfortable when testing , to minimize the number of underlying bugs, to make the product development process more agile, to shorten the product development cycle, and finally, after the product is launched, to allow users to use it more smoothly and to make users feel that the product service has zero defects.
Different from unit testing, interface testing is essentially a black-box testing, so it is very suitable for full-time test engineers to participate and cover.
Interface test framework selection
1. At present, the most common method for the selection of interface testing frameworks is to use UI-based interface testing frameworks such as jmeter, soapUI, postman, and robotframework.
The advantage is that business testers do not need or rarely write test code, and the barriers to entry are low. In the past few years, many companies have developed similar test frameworks, with front-end and back-end, full-time test developers maintain, and business testers only You need to know how to operate without participating in specific coding.
This method seems very lofty, but the actual problem is that the main work in the implementation process becomes the maintenance of the test framework, which relies heavily on the design and development capabilities of full-time test developers. Every time a new interface protocol is added (such as dubbo, hessian or internally customized protocols) need to add support to the framework; what is more fatal is that once the core test developers flow, it is easy to cause the collapse of the entire interface test system; It's not fair. I have personally interviewed too many engineers who only use the XXX test framework of a large company but don't understand the specific implementation method at all.
It has long been predicted in "The Way of Google Software Testing" that confidentiality and privatization of basic testing facilities cannot obtain the benefits imagined. This method means expensive and slow, and it is difficult even between different projects within the company. Do reuse. The future testing infrastructure must be based on shared code and open source frameworks, and test developers need to make more use of and contribute to open source projects.
Recently, I re-read this book that almost changed the software testing industry four or five years ago, and found that the predictions in it are so accurate. Of course, it can also be considered that the entire domestic industry is evolving in the same way as Google.
2. Use java interface frameworks such as junit and testng to directly write test codes for testing, and at the same time abstract and establish basic libraries or methods for some repetitive tasks.
It is a bit similar to unit testing. This method is scalable and flexible. As a programmer, you can use code to implement flexible scene organization and functions. You only need to do a little secondary development, but test engineers need to have a certain coding foundation.
It was quite difficult to implement this method in the past few years, because there are very few test engineers who understand java code in the market, but it is not as difficult to implement today as there are more and more requirements for the development capabilities of test engineers. Difficulties, coding skills in languages such as java/python have also become the basic requirements for our team recruitment.
In addition, the reason why java is used here instead of other languages is mainly because the team’s technical reserve java is its strength, it has a rich open source test library, and the products of general Internet companies are basically developed using the java framework, and the development team’s technology It is very necessary to keep the stack consistent.
Our encapsulated interface testing framework
Many companies have made various interface frameworks, all of which are designed and developed based on their own company's business foundation. Based on our own business characteristics (in order to avoid suspicion of advertising, I try not to mention specific company information), we also encapsulate our own interface testing framework gtest-framework, which is gradually being used in developer unit tests.
What gtest-framework has to do:
1. Pre-data preparation and automatic cleaning.
2. Implementation and encapsulation of common interface protocols.
3. Support for dependency injection configuration.
4. Integration of various general processing methods such as files, pictures, xml, characters, etc.
5. Expansion of the assertion method, etc.
Key Practices of Interface Testing
1. Data preparation
The data preparation of the interface test generally refers to the data preparation of the database, and sometimes includes the data preparation of files and caches. The specific implementation can be considered from the following aspects
(1) Prepare test data in a hard-coded way. When writing test code, insert whatever data is used. In order to avoid data duplication, many people are used to using random characters or random numbers (this method may cause instability of test cases, try to avoid it).
(2) The test data can be prepared directly by calling other APIs. This is more useful when testing the top-level service. For example, to test the purchase of goods, you need to prepare the data of the goods to be purchased and the user data of the goods to be purchased. At this time , you can directly call the api for generating products and the api for generating users to directly generate test data. This method is simple to implement, but the premise is that the corresponding api is required and the function of this api is correct.
(3) Use excel or xml to prepare test data. This method of preparing test data is mainly aimed at the preparation of object data. For example, you can map a piece of commodity data to a piece of data in excel, because generally development will use pojo mapping, and in preparation When testing data, the setting of these pojo object properties is often repetitive and heavy workload. Using excel or XML to prepare these data can reduce the repetition of preparing these data in the code.
Generally, we use 2/3 methods, 3 of which mainly use the characteristics of dbunit, spring-test, unitils and other test frameworks to add custom annotations through secondary development, and easily import files in excel or xml format And automatically rollback the data after the test is completed.
/**
* @ClassName: TestJdbcDataSet
* @Description: 采用自定义TestDataSet注解方式准备测试数据，推荐。
* @author Cay.Jiang
* @date 2017年7月10日 上午9:10:29
*
*/
public class TestJdbcDataSet extends BaseCase{
Map<String, Object> args = new HashMap<String, Object>();
@Test
@TestDataSet(locations={"/tmp/domaininfo.xls"},dsNames={"mysqlDataSource"})
public void test01_mysql(){
args.put("selfdomain", "baidupc2");
List<Map<String, Object>> result=JdbcUtil.queryData(mysqlJdbcTemplate, "domaininfo", args);
System.out.println(result);
assertEquals("合作商接入名称",result.get(0).get("remark"));
}
}
Refer to /tmp/domaininfo.xls in the above code: domaininfo.xls, where the Excel format uses Sheet as the table name, the first line defines the field name, and the rest of the lines correspond to the data.
Multiple datasets:
@TestDataDataSet(locations={"Data1.xls","Data2.xls"},dsNames={"dsNameA","dsNameB"}), the data of Data1.xls will be inserted into the database pointed to by dsNameA , the data of Data2.xls will be inserted into the database pointed to by dsNameB
2. Assertion
Common assertion methods include Assert and Hamcrest that come with JUnit. The assertion method that comes with JUnit has very limited functions and can only meet the most basic needs. Hamcrest is relatively feature-rich, but the library has not been updated for many years. Moreover, Hamcrest, like the assertion method that comes with JUnit, has a fatal shortcoming, that is, when there are multiple assertions in a case, if one of the assertions fails, the assertions after it will not be executed. Here I recommend a new assertion artifact AssertJ to everyone.
AseertJ: Known as a streaming assertion. What is stream mode? A common asserter statement can only assert one checkpoint on the actual value, but AseertJ supports one assertion statement to assert multiple checkpoints on the actual value at the same time, which makes the assertion statement more concise and suitable for reading. AseertJ also supports executing all assertions at one time, and then collects all failed assertions for feedback. Of course, in addition to this, AseertJ has many other features, you can refer to the official documentation to dig slowly. The following will illustrate the advantages of AseertJ:
public class TestCase extends BaseCase{
UserProfileBO user = new UserProfileBO();
@Before
public void init(){
user.setAddress("杭州");
user.setMobile("1386800000");
user.setUserName("测试账号");
}
/*
* JUnit 内置的断言
*
* 1、其中一个断言失败后，后面所有断言将不会执行。
* 2、支持的断言方法较少
*
*/
@Test
public void testAssertJUnit(){
Assert.assertEquals("地址",user.getAddress(),"宁波");
Assert.assertEquals("手机",user.getMobile(),"13868000000");
Assert.assertEquals("手机",user.getUserName(),"测试");
Assert.assertNotNull(user.getMobile());
Assert.assertTrue(user.getMobile().startsWith("138"));
Assert.assertTrue(user.getMobile().length() == 11);
}
/*
* Hamcres 断言
*
* 1、其中一个断言失败后，后面所有断言将不会执行。
* 2、支持的断言方法丰富，但是已经多年不更新。
*
*/
@Test
public void testHamcrestMatchers() {
MatcherAssert.assertThat(user.getAddress(), equalTo("宁波"));
MatcherAssert.assertThat(user.getMobile(), equalTo("13868000000"));
MatcherAssert.assertThat(user.getUserName(), equalTo("测试"));
MatcherAssert.assertThat(user.getMobile(), allOf(is(nullValue()),startsWith("136")));
}
/*
* AssertJ 断言
*
* 1、支持所有断言执行后，失败断言统一反馈。
* 2、支持的断言方法丰富。
* 3、支持流式断言，方便阅读。
*
*/
@Test
public void testAssertJ(){
//断言集合，执行所有断言后，失败断言统一反馈。
SoftAssertions.assertSoftly(softly -> {
softly.assertThat(user.getAddress().equals("宁波"));
softly.assertThat(user.getMobile().equals("13868000000"));
softly.assertThat(user.getUserName().equals("测试"));
});
//流式断言
Assertions.assertThat(user.getMobile())
.isNotNull()
.startsWith("136")
.hasSize(11);
}
}
3. Jenkins integration interface test
(1) Set the warehouse address and identity information of the test code
(2) Set maven operating parameters
If you want to execute some interface use cases, you can execute the specified case through -Dtest=XXX (test class name), and multiple class names are separated by commas ","
(3) Set the job execution mechanism, the following figure shows that it is executed regularly every day
4. Pipeline parameter injection
As mentioned earlier when writing the pipeline, our interface test code needs to support external parameter injection, such as the service address of the test. Different branch codes may be deployed on different test servers, and parameters need to be passed in the pipeline. The standardized way drives the interface testing of different servers.
Here we can use maven's -D (Properties attribute) to achieve, as follows:
(1) dubbo uses the properties configuration file, but the specific parameters are packaged and replaced by ${key} placeholders
(2) Specify the parameter values in the corresponding configuration file in the pom file of maven (the parameter values specified here will be overwritten by the parameter values passed through maven -D)
Note here: You also need to start the filter filter of resources
(3) Use the maven command line to set the property value
And parameterize the value to support pipeline parameter passing
5. Pipeline code implementation
stage('接口自动化测试') {
steps{
echo "starting interfaceTest......"
script {
//为确保jetty启动完成，加了一个判断，确保jetty服务器启动可以访问后再执行接口层测试。
timeout(5) {
waitUntil {
try {
//确保jetty服务的端口启动成功
sh "nc -z ${serverIP} ${jettyPort}"
//sh "wget -q http://${serverIP}:${jettyPort} -O /dev/null"
return true
} catch (exception) {
return false
}
}
}
//将参数IP和Port传入到接口测试的job，需要确保接口测试的job参数可注入
build job: ITEST_JOBNAME, parameters: [string(name: "dubbourl", value: "${serverIP}:${params.dubboPort}")]
}
}
}
Test code specification (for reference only)
1. Test project naming convention
Interface test:
generally requires an independent test project, the naming rule of the test project is: "test-" + the name of the project to be tested, such as test-kano
unit test:
no need to rebuild the independent test project, and develop code Put them in the same project.
2. Test directory definition specification
The test code is placed under the "src/test/java" of the test project.
The test configuration files are uniformly placed under "src/test/resources".
3. Package name definition specification
Consistent with the package name in the project under test
4. Test class naming convention
The test class naming rule is: start with Test and end with the name of the object to be tested, for example,
Test+ business to be tested, Test+ The interface to be tested, Test+ the class to be tested
Another way is: end with Test and start with the name of the object to be tested, such as the
business to be tested+Test, the interface to be tested+Test, the class to be tested+ Test
depends on personal habits. For the convenience of case positioning, the current test team generally uses the first method.
5. Test case naming conventions
The naming rules of test cases are: test+ case operation_condition state, use the lowerCamelCase style uniformly, and must follow the hump form.
The word convention is the same as the test class name
6. Common constraints of interface test code
(1) Data cleaning and construction
@BeforeClass @Before Do data preparation and other related operations --before loading the test class, you need to load the common scene data of all test cases, and at the same time load special test data @AfterClass @
when running a single test case
Do test data cleaning and other related operations in After
--Clean up the use case site after executing relevant tests
(2) Assertions
--Don't make unnecessary assertions
In test mode, sometimes you can't help but abuse assertions. This approach will lead to more maintenance Difficult and should be avoided as much as possible. Only explicitly test the characteristics indicated by the test method name, because for general code, it is an important goal to ensure that the test code is as small as possible-the use of explicit assertion should always be used in preference to
assertEquals
( a, b) instead of assertTrue(a == b), because the former will give a more meaningful test failure message. This rule is especially important in cases where the input values are not determined in advance - the argument order of the assertion should be
appropriate
(3) Test cases are kept independent
--ensure that the test code is independent of the project code
--in order to ensure that the test is stable, reliable and easy to maintain, there must be no interdependence between test cases, nor the order of execution.
(4) Test code should consider error handling
- if the previous code execution fails, subsequent statements will cause the code to crash, and the rest of the test cannot be executed. Be prepared for test failure at any time to avoid a single failed test item interrupting the entire test Execution of the suite
-- don't write your own catch code block, that is, only the case of test failure, there should be no catch situation
As someone who has been here, I also hope that everyone will avoid some detours
Here I will share with you some necessities of the way forward in automated testing, hoping to help you.
(software testing related materials, automated testing related materials, technical questions and answers, etc.)
I believe it can make you better progress!
Click on the small card below
