(1) How to carry out interface automation?
0. Research, premise preparation and thinking
a) Premise:
1. When formally designing use cases, combine tools such as postman/jmeter
2. Design different test data, initiate requests, and check whether the response results are consistent with the design
3. (Required Go through manual testing) -- found bugs
b) storage method of use cases:
1. excel form - configure json path
2. json file - there are many request parameters, written in json file
3. yaml file - httprunner3.0
4 , Database - create tables
c) How about automation coverage
coverage: functional/manual use case coverage - 30% - 90%
1. How long have you been automating the interface of this project?
2. How big is the system? Is it complicated?
3. How many coverage use cases? What difficulties (and solutions) did you encounter?
1. Requirements analysis
Understand the business functions of the project, modules with more bugs, what are the relatively stable interfaces, and what are the core functions
2. Understand the interface
2.1 Look at the interface by capturing packets
2.2 Understand through the interface document
3. Selection of automation framework and tools
3.1 Scalability of work and extension language + select several complex interfaces for trial use
3.2 Comparison of framework structures
3.3 Specification naming
4. Write interface use cases
4.1 Write interface use case scripts
- 1. The core business goes first - the highest user utilization rate
- 2. Use case priority - common functional scenarios/required parameters
- 3. The format validity of the parameter - no verification is done on the backend
- 4. Normal use cases are designed first, abnormal scenarios (comprehensive)
4.2 Join jenkins integration as soon as possible
4.3 Regularly report the progress
4.4 Test reports, automatically send reports, analyze the cause of use case failure
4.5 Record the interface automation from the beginning to the current bug
4.6 Exception handling
5. Persistence layer structure
1. Insert data directly into the database
6. Maintenance phase
1. Develop and modify interfaces, test synchronously modify interface scripts
2. Add new interfaces, synchronously add new interface use cases 3. Optimize
scripts and daily frameworks
(2) How to test a single module?
Single module test: It is mainly used in the test work to check the interface implementation of a single business function, or to debug test data.
Step 1: Sort out the upstream and downstream call chains
1) Why do we need to sort out the upstream and downstream call chains?
At present, the back-end services of Internet products are basically deployed in a distributed manner . An interface may call other interfaces, and it may also be called by other interfaces. There are innumerable dependencies between interfaces.
It is obviously impossible to do a good job of interface testing if you just adjust the parameters alone. (Developers can adjust (tiao) interface parameters themselves, what else do they need to test?)
2) How to sort out the upstream and downstream call chains?
1. Look at the project wiki, product documents and development documents
2. Look at the code written by the developer, read the code
3. Sort out the relationship between upstream and downstream calls, and draw a system flowchart. If there are still unclear points, you can find PM, developer communication confirmation
Step 2: Write interface test cases
If you want to automate the interface , the interface test case is also very helpful.
Here is an example of an interface test case:
Step 3: Test interface document & debug interface
At the beginning of project development, front-end development and back-end development will jointly agree on a set of interface specifications, and then the back-end development will write the interface documents, and then the front-end and back-end can carry out collaborative development according to the agreement.
There are several ways to manage and edit interface documents :
- Some teams are accustomed to using wiki or online documents to write interface documents;
- Some teams like to use professional interface documentation tools, such as: Swagger, Yapi , etc. to generate interface documents.
The test interface documentation can refer to the following test points:
- To ensure that development must provide interface documentation . If the development has no habit of writing interface documents, you should push the development to write interface documents.
- Check whether the format and content of the interface document are complete , including: URL, request method, header, input parameters, return value, sample demo, etc.
- Check whether the interface design complies with company specifications . Including interface naming, interface format, field naming, field type, response status code, interface fault tolerance, whether the field is redundant, whether the interface is authenticated, whether to distinguish versions, etc.
When the back-end development completes the development of the interface, we can start preliminary testing of the interface in advance.
Proceed as follows:
- Deploy the backend code to the test environment .
- Use postman or swagger to test the interface mentioned in the interface document.
Step 4: Front-end interface test & Mock data (interface level test)
The previous steps are just to use the test tool to initiate a network request to simulate the interface call.
But in a real scenario, the interface of the search gateway is actually provided for APP/WEB/small program to call.
We also need to pay attention to whether the front-end calling process is normal . (You need to wait for the front-end development to be completed before you can intervene in the test)
You can use Charles to capture the requests sent by the front end,
- Verify that the parameters passed to the front-end calling interface are correct;
- Verify whether the interface response of the backend meets expectations;
- After the front end gets the data, whether the interaction and UI display are correct.
When some data has multiple states and the data is difficult to construct, we can use Mock data to test.
The common ways of mocking data are :
- Use Fiddler or Charles to tamper with requests and responses.
- If it is a dynamic language such as PHP or Python, you can directly change the conditions in the back-end code.
- To modify data in the database.
- Use professional Mock tools to construct data, such as: EasyMock, TestableMock, Mockjs, etc.
The faster way , of course, is to directly use Charles to simulate .
Step 5: Back-end interface testing & business logic coverage (see logs, codes)
see log
During business testing, we need to keep an eye on the backend log status.
Sometimes the interface response data is normal, but the backend log may be reporting an error,
look at the code
It is recommended that you must read the source code of the development when doing interface testing.
Reading the source code can give you a deeper understanding of the business logic implementation.
What if the amount of code is large?
Let me tell you a little trick: After the development submits the code, we can view his Commit record on Gitlab , or make a diff between his development branch and the branch of the production environment , so that we can know what he has changed.
Step 6: Interface performance tuning (Arthas)
Troubleshooting process :
(1) First try to reproduce it on the APP
(2) Step by step to troubleshoot the reason for the slow response of the interface through the trace of Arthas:
enter the Arthas command line
java -jar arthas-boot.jar
The method called by the trace interface
trace 类名 方法名
Step 7: Interface version control & diffy
Generally, the interface will distinguish between versions. If the interface is not very standardized, or some general logic has been changed, a regression test on the old version is required at this time.
The most stupid way is to compare and test the old and new versions of the two apps. We can also use the diffy tool for regression testing.
Step 8: Start to automate the interface
Interface automation is generally used in scenarios such as online inspection regression and smoke test.
To achieve interface automation, use the following methods:
coding：
python+pytest+requests is currently used in this way. (Small and beautiful, easy to customize)
(3) How to test multiple module associations?
Module association: refers to the dynamic association of the input and output parameters of two or more related APIs in the form of parameterization, so as to realize the test coverage of the entire transaction and achieve the basic tool interface automation test.
Step 1: Sort out the upstream and downstream call chains
1) Why do we need to sort out the upstream and downstream call chains?
At present, the back-end services of Internet products are basically deployed in a distributed manner . An interface may call other interfaces, and it may also be called by other interfaces. There are innumerable dependencies between interfaces.
It is obviously impossible to do a good job of interface testing if you just adjust the parameters alone. (Developers can adjust (tiao) interface parameters themselves, what else do they need to test?)
2) How to sort out the upstream and downstream call chains?
1. Look at the project wiki, product documents and development documents
2. Look at the code written by the developer, read the code
3. Sort out the relationship between upstream and downstream calls, and draw a system flowchart. If there are still unclear points, you can find PM, developer communication confirmation
Step 2: Write interface test cases
If you want to automate the interface , the interface test case is also very helpful.
Here is an example of an interface test case:
Step 3: Test interface document & debug interface
At the beginning of project development, front-end development and back-end development will jointly agree on a set of interface specifications, and then the back-end development will write the interface documents, and then the front-end and back-end can carry out collaborative development according to the agreement.
There are several ways to manage and edit interface documents :
- Some teams are accustomed to using wiki or online documents to write interface documents;
- Some teams like to use professional interface documentation tools, such as: Swagger, Yapi , etc. to generate interface documents.
The test interface documentation can refer to the following test points:
- To ensure that development must provide interface documentation . If the development has no habit of writing interface documents, you should push the development to write interface documents.
- Check whether the format and content of the interface document are complete , including: URL, request method, header, input parameters, return value, sample demo, etc.
- Check whether the interface design complies with company specifications . Including interface naming, interface format, field naming, field type, response status code, interface fault tolerance, whether the field is redundant, whether the interface is authenticated, whether to distinguish versions, etc.
When the back-end development completes the development of the interface, we can start preliminary testing of the interface in advance.
Proceed as follows:
- Deploy the backend code to the test environment .
- Use postman or swagger to test the interface mentioned in the interface document.
Step 4: interface scene design
Background : The existing platform is relatively mature for the single-service single-interface automated test process, and the demand feedback for complex cross-service automated use case configuration is increasing. Therefore, support for complex test scenarios is added.
What kind of use case is suitable for scenario
? It can also be called business flow, for example: take an e-commerce case, order purchase-payment-verify payment status-view permissions-refund-verify refund status
- Regardless of whether it is cross-application or not, steps have strong dependencies before and after
- Core business scenarios, such as order-payment-performance-refund
Based on the above background, the corresponding function points of this project are as follows:
- Add the concept of [Scene Set], which is equivalent to the original [Project]
- Add the concept of [Test Scenario], which is similar to the original [Use Case Set]
- Trigger associated test scenarios
Step 5: Front-end interface test & Mock data (interface level test)
The previous steps are just to use the test tool to initiate a network request to simulate the interface call.
But in a real scenario, the interface of the search gateway is actually provided for APP/WEB/small program to call.
We also need to pay attention to whether the front-end calling process is normal . (You need to wait for the front-end development to be completed before you can intervene in the test)
You can use Charles to capture the requests sent by the front end,
- Verify that the parameters passed to the front-end calling interface are correct;
- Verify whether the interface response of the backend meets expectations;
- After the front end gets the data, whether the interaction and UI display are correct.
When some data has multiple states and the data is difficult to construct, we can use Mock data to test.
The common ways of mocking data are :
- Use Fiddler or Charles to tamper with requests and responses.
- If it is a dynamic language such as PHP or Python, you can directly change the conditions in the back-end code.
- To modify data in the database.
- Use professional Mock tools to construct data, such as: EasyMock, TestableMock, Mockjs, etc.
The faster way , of course, is to directly use Charles to simulate .
Step 6: Back-end interface testing & business logic coverage (see logs, codes)
see log
During business testing, we need to keep an eye on the backend log status.
Sometimes the interface response data is normal, but the backend log may be reporting an error,
look at the code
It is recommended that you must read the source code of the development when doing interface testing.
Reading the source code can give you a deeper understanding of the business logic implementation.
What if the amount of code is large?
Let me tell you a little trick: After the development submits the code, we can view his Commit record on Gitlab , or make a diff between his development branch and the branch of the production environment , so that we can know what he has changed.
Step 7: Interface performance tuning (Arthas)
Troubleshooting process :
(1) First try to reproduce it on the APP
(2) Step by step to troubleshoot the reason for the slow response of the interface through the trace of Arthas:
enter the Arthas command line
java -jar arthas-boot.jar
The method called by the trace interface
trace 类名 方法名
Step 8: Interface exception mechanism (Chaosblade)
Because the interface depends on many services, it is often necessary to call other interfaces. If there is an exception in the dependent service, we need to consider whether our interface is fault-tolerant or downgraded.
You can use Chaosblade to inject exceptions. (not required, but better)
Step 9: Interface version control & diffy
Generally, the interface will distinguish between versions. If the interface is not very standardized, or some general logic has been changed, a regression test on the old version is required at this time.
The most stupid way is to compare and test the old and new versions of the two apps. We can also use the diffy tool for regression testing.
Step 10: Start interface automation
Interface automation is generally used in scenarios such as online inspection regression and smoke test.
To achieve interface automation, use the following methods:
coding：
python+pytest+requests is currently used in this way. (Small and beautiful, easy to customize)