Editor's note: The author of this article is Lin Yonglong of the Internet retail platform Pupu Supermarket.
Pupu Supermarket is a mobile Internet shopping platform with instant delivery within 30 minutes. The commodity categories in the supermarket mainly include: fruits and vegetables, meat, poultry, eggs and milk, grain and oil seasoning, alcohol and beverages, leisure food, personal care, cosmetics, cleaning supplies, daily necessities, etc. together. Users can get their daily needs anytime, anywhere on their mobile phones without leaving home.
Pupu Supermarket (hereinafter referred to as Pupu Technology) hopes to create a new generation of B2C e-commerce platform with an innovative business model and efficient warehousing and distribution model that surpasses the existing mainstream e-commerce websites in terms of customer experience. Taking technology as the core driving force, improving efficiency and reducing costs; saving time and money for users with a 30-minute delivery speed and lower price, creating the ultimate experience; promoting the improvement of China's e-commerce environment with reliable quality, and promoting Chinese food The security process has become an excellent Internet company that is loved by consumers and contributes to social value.
In order to meet the company's rapidly developing business needs, the testing team of Pupu Technology is constantly exploring the road of improving quality and efficiency. The testing team realized early on that simply increasing the manpower to improve the delivery speed of business tests is no longer a good choice, and the continuous increase in business complexity has also brought new challenges to the testing work.
In order to ensure the delivery speed of the requirements, in addition to the conventional functional testing methods, we also carry out automated testing like most enterprises in functional testing. Among them, the automated testing of interfaces is realized through the combination of "Java+TestNG+Jenkins" and other tools. . After more than half a year of implementation, the continuous integration test of the online P0 interface was also completed.
However, we soon found that the proficiency of test development for business is always unsatisfactory. How to make every use case request of automated testing have rich business logic has become a new pain point in testing work.
To this end, we investigated the mainstream automated testing platforms, and finally chose the MeterSphere one-stop open source continuous testing platform that seems to be the most likely to be learned quickly, efficiently, and easily.
Test status and thinking
The testing team of Pupu Technology has a scale of nearly 70 people, which is affiliated to the R&D center of Pupu Group. There are more than 30 parallel testing projects in each product line, all of which are based on the microservice architecture. It mainly includes the test content of App, function, interface, UI, performance, Elasticsearch, big data, operation and maintenance platform, middleware and so on. Due to the short project delivery cycle, heavy tasks and tight testing time in daily work, it is difficult to achieve 100% test coverage and regression.
After considering these factors, compared with the existing test content with higher practical costs such as unit testing and system testing (UI, integration, etc.), we choose to incorporate automatic interface testing into the job functions of functional testing students, and use accurate testing to Guide regression testing, use interface automation to increase test frequency, and use coverage to identify test effectiveness.
So the speed of production for automation use cases is something we have to address. According to the existing strategies, there are two choices in front of us: one is to develop a platform by ourselves, and the other is to purchase SaaS services or refer to open source platforms. In view of the self-research experience of other test platforms, we were finally moved by the use case writing function of MeterSphere.
■ Business project: Sort out P0 interfaces and scenarios from the online traffic monitoring platform, core business and core processes, and use Chrome's F12, Charles, Postman, JMeter, TestNG, HttpRunner and other tools to manually capture and Verify the request and response assertion of the external interface of the page by recording the script;
■ Operation and maintenance projects: Test the interface of the company's internal and external operation and maintenance systems. Such projects are large in number and short in cycle, and each project interface design scheme is different. Major online outage. Under normal circumstances, we basically carry out automated testing guarantees for UT and FT through Code.
In addition, each project has tens of thousands of external HTTP interfaces (excluding Dubbo, SQL, Elasticsearch, etc.), and the HTTP interfaces need to be maintained uniformly through Swagger. In addition, the project also needs to conduct single-interface testing, multi-interface testing based on business scenarios, and long-term iterative maintenance of interface test cases and scripts.
Pain points of daily testing work
The Pupu technology test team has certain difficulties in multi-person collaboration, as well as collaboration between testing and development, operation and maintenance, and products. MeterSphere one-stop open source continuous testing platform can solve this problem. For us, MeterSphere's one-stop continuous testing capabilities are very attractive.
In the daily testing work, the status quo of interface testing performed by each test group is varied, both in use case management methods and strategies, and this situation even occurs only for the same business product. For example, the testing work is carried out based on the HttpRunner framework, and some are carried out based on JMeter and Postman. It is difficult to standardize and integrate all kinds of test scripts, and it is impossible to carry out unified management. In addition, the personnel effectiveness and product quality between projects cannot be reasonably measured and horizontally compared, and continuous construction is even more difficult to achieve.
At the same time, the execution of the interface test case was not completed in the self-test link of the development phase. We hope that the testers will write the interface test cases, and even cultivate the developers to write them by themselves, and let the developers complete the test access, and the testers will conduct the final acceptance after passing. Rather than having the tester repeatedly execute the use case and then notify the developer of the result. This seems to be a simple link, but without the support of a unified platform, only relying on Postman, Git, Jenkins and other tools to maintain use case scripts, it is difficult to achieve seamless connection between development and testing.
In addition to the requirements for use case writing, our interface test platform also needs to be connected to other test platforms, such as precision testing, CI/CD, etc., and the interface automation test needs to be connected to various research and testing processes and scheduling tools.
Advantages of MeterSphere Continuous Testing Platform
After investigating the SaaS platforms of some major manufacturers and some open source platforms, we found that the MeterSphere open source continuous testing platform is very easy to use, and the test team is easy to use. With the deepening of use, we also found more and more advantages of MeterSphere, which basically include the following advantages:
1. Low switching cost and easy to use
Whether it is for traditional manual functional testing or for automated interface testing, the ease of use design of the MeterSphere platform is very good. Especially in the part of use case writing, students who have one year of experience in Internet product testing can get started quickly. It took only one month for our testing team to test MeterSphere to actually output more than 2,000 test cases;
2. Excellent scenario use case writing function
At present, we have realized the interface test platform linkage CI/CD research and development process, which can easily complete the overall test closed-loop of accurate test recommendation, interface automation and coverage, and output the admission/acceptance test report.
① Interface definition: Quickly implement interface definition through Swagger, and can realize timing synchronization, even the import function of one-click synchronization;
② Single-interface interface test case: MeterSphere supports the general capabilities of defining various requests, parameterization, association, and assertion, especially JSONPath's quick assertion and List extraction parameter transfer is more efficient; at the same time, it extends the pre-script and post-script. , SQL, parameter extraction, global settings and more tools, support BeanShell, Python and other scripting languages. It is a pity that Python only supports basic libraries, but even so, MeterSphere can basically meet the vast majority of interface test scenarios of various projects;
③ Scenario interface testing: In addition to completing the forward and reverse routine interface testing of a single interface, we also need to make effective and meaningful interface requests according to business scenarios to improve the quality of use cases. Scenario use cases are relatively complex, and basic scenarios, databases, and some third-party interfaces need to be preset, and the timeliness of the business must also be considered. In this regard, MeterSphere performs well. Not only can it support interface and scene references, various types of commonly used controllers, and custom requests, but also scripts are sufficient to write more than 80% of complex scenes.
④ Flexible use case tags: In addition to the definition of levels and categories for use cases, MeterSphere also supports the function of custom tags, which can well complete use case filtering for various test strategies.
The implementation of CI/CD
The MeterSphere open source continuous testing platform can effectively improve the production speed of automated test cases, and improve the continuous delivery capability of software through automated testing.
After the introduction of MeterSphere, we tried to write interface test cases in parallel by testers, integrate the company's various platforms to schedule automated tests through the CI/CD process, and let the developers execute the automated test plan by themselves. Repeated communication to realize the left shift of the test. In order to realize the test left shift, the developed interface design specification is also indirectly standardized, otherwise the test case cannot be written and the execution pass rate is low.
In addition, through "accurate + automated testing + coverage testing strategy", we have improved the test efficiency and also increased the release frequency of test-free requirements.
Looking forward to improvements
Finally, let’s talk about some expectations and suggestions for improvement of the MeterSphere open source continuous testing platform. I hope this open source project can do better and better:
■ Support Python3 as early as possible. Most tester practitioners basically choose Python3 as the entry language. If MeterSphere supports Python3 for data conversion of use cases, it will greatly improve the ease of use of the platform and the number of users;
■ The controllers supported by the scenario use case can refer to Pytest, such as Skip, Re-runs, fail exit and other functions;
■ Optimize report readability. On the one hand, the report is prone to step confusion when referencing multiple nested scenarios; on the other hand, it is very time-consuming to locate errors in scenarios with many steps. For details on this, please refer to the steps in the Allure report.