The landing of R2 in the omni-channel business line | JD Cloud technical team

With the growth of the business and the high-frequency iteration of the system, the quality assurance work urgently needs to introduce more scientific and efficient testing methods to help the high-quality delivery of the business. During the first-phase test of the Great Wall project, the omni-channel quality team introduced R2 technology from the technology platform department, which greatly improved the quality of project delivery. Therefore, this article will focus on how omni-channel quality teams use R2 to ensure business quality.

1. Why introduce R2?

The Omni-channel Great Wall project sinks the existing transaction capabilities of the omni-channel to the transaction platform, and realizes the deep integration of online and offline systems at the technical system level. through) laid the cornerstone, running through the entire order life cycle from omni-channel stores, merchandise, purchases, promotions, coupons, general ordering, payment to after-sales. The project involves as many as 27+ applications, 300+ interfaces, and a wide range of business. If you completely rely on traditional manual testing, not only will you need to invest a lot of manpower in a limited time, but it will also be difficult to ensure that each iteration returns to cover all business scenarios, which will eventually affect the quality of project delivery.

Therefore, in order to more comprehensively and effectively cover all online business scenarios, the Omni-Channel Quality Team researched platform tools suitable for data comparison before and after the Great Wall project transformation, and finally chose the R2 tool of the Technology Platform Department among various tools. The full name of R2 is Record and Replay, that is, recording and playback. After the test colleagues communicated with the R2 team and tried it out, they found that it is an auxiliary testing tool that is very suitable for large-scale project comparison of large amounts of data. In addition, the access of R2 is also very simple. You only need to access the required pfinder version in the application, and R2 can find the application to be tested and perform auxiliary tests. By recording real online traffic and playing it back online or in a test environment, it can not only help test and improve business test cases, but also shield the external influence on the code to be tested by mocking the upstream jsf interface, mysql database, and jimdb cache. , only focus on code changes inside the system.

Based on the above positioning and characteristics of R2, R2 can fully assist the technical improvement test and regression test of the system. The Great Wall project introduced in this article belongs to the technical transformation test.

2. How does R2 enable omni-channel ecological business testing?

First, record online traffic on relevant interfaces. Then, asynchronously playback to the application under test. Finally, according to the diff results of the online interface and the application under test, as well as the recorded related requests, comprehensively analyze and evaluate whether the system has reached the online standard. Taking the promotion business of the Great Wall project as an example, the introduction is as follows.

2.1 Recording and playback

Since the promotion business of the Great Wall project has high real-time requirements for use cases, real-time DIFF is used to facilitate regression testing. The main difference between real-time DIFF and offline DIFF is: offline is to record the online traffic first, not to play it back temporarily, and then play it back when it needs to be played back. And real-time is to start asynchronous playback immediately after the recording is completed, thereby reducing the impact of timeliness on test results. The recording and playback tasks corresponding to real-time DIFF need to pay attention to the following three points:

Task settings: set the task name, select the playback environment, and select the application to be tested. As shown in the upper left corner of Figure 2.1.

Link settings: First, you need to select the interface to be DIFF in real time, and each interface should enable interface playback and select the interface alias to be played back. Note: This alias is the alias for the interface to request again after the subsequent traffic recording. Then set the rules related to the interface response. Among them, there are three types of interface impact comparison rules, the first and second are commonly used, and the third type needs to write a comparison script by yourself. The equals method of the request comparison rule refers to the comparison of the entire method, and all fields in the method will be compared. Visual configuration refers to the fields that need to be set to be ignored. If there is a list type in the response result, the sorting of the fields in the list needs to be set to ensure the accuracy of data comparison. Some examples are shown in Figure 2.1.

Figure 2.1

Application setting: refers to the machines selected for recording, and the traffic recorded on these machines will be played back asynchronously to the application under test. As shown in the lower right corner of Figure 2.1, the selected machine is the corresponding IP of the machine.

2.2 Analysis of results

After the real-time DIFF task playback is completed, click the result to see the recorded requests of this task and the number of DIFF requests. If there is a DIFF task, you can compare the results to see which method has different request results. As shown in Figure 2.2.

Figure 2.2

Take the real-time DIFF result of an interface for promotion as an example. Figures 2.3 and 2.4 respectively query the interface request parameters and response results of the corresponding sku in a certain seckill session. The interface input parameters in Figure 2.3 mainly include the store id, session and tenant, and each sku in the response results contains the corresponding sku Promotion id, promotion price and purchase limit information, where boundNumber represents the purchase limit of the event, and surplusRepertory represents the inventory of the event.

Figure 2.3

The left side of Figure 2.4 is the current request result, and the right side is the historical recording result. It can be found that the values corresponding to the same promotional plusRepertory of the same sku in the left and right results are different. In this regard, the reasons for the differences are analyzed as follows:

(1) First of all, considering that there will be a certain time difference in asynchronous playback, the remaining inventory of real-time query may be inconsistent. However, the corresponding remaining inventory in the historical version is 0, and the historical version is the first to request online. This request is earlier than the request on the left. If it is due to the time difference, it will only appear that the remaining inventory on the left is smaller than the one on the right. Yes, so the reason for the diff is not caused by the time difference.

(2) Judging from the DIFF results, the problem is that the remaining inventory queried by the application under test is different. Through analysis, it is found that the remaining inventory corresponding to each sku on the left is equal to the active inventory, and there is no corresponding change in the remaining inventory due to orders. Then, through the simulation positioning of the same scenario, it is found that the application under test does not dynamically change the remaining active inventory for the scenario where the active inventory is set, which will cause the SKU to be oversold and affect business operations.

Figure 2.4

3. What are the benefits of R2's omni-channel empowerment?

3.1 Access and usage of R2 in all channels

Omnichannel began to access R2 from December 2021. Under the guidance of the R2 team, the access of 27 core applications and 320 interfaces was quickly completed within 1 day, and currently accesses about 79+ applications. The Great Wall project will use R2 for real-time DIFF and offline DIFF during the period from its launch in December 2021 to February 2022. At the same time, the core interface will also use real-time DIFF testing during the subsequent flow switching process and after the flow switching is completed, so that The stability of the system is guaranteed.

Figure 3.1

Figure 3.1 shows the development of omni-channel applications connected to R2. The abscissa represents time, and the ordinate represents the number of applications that have been connected to R2 in every month. The Great Wall will go online at the end of December 2021. In order to better monitor the quality of the Great Wall project, the omni-channel core application will be connected to R2 one after another this month, and R2 will be used to record online traffic and playback it to the application under test for early monitoring and testing application, discover inconsistencies in advance, and repair and optimize them early. Because most of the applications in the early stage were connected to R2, there is no obvious change in the newly added applications in the later stage. After the slicing is completed in February 2022, some requirements will be added in March 2022, and then this part of the requirements will continue to be connected to R2 for testing.

Figure 3.2

Figure 3.2 shows the number of tasks for real-time DIFF and offline DIFF using R2 in Omni-channel. The abscissa indicates time, and the ordinate indicates the number of tasks. Blue indicates real-time DIFF tasks, and orange indicates offline DIFF tasks. December 2021 is the initial stage of connecting various applications to R2, so there are many real-time and offline DIFF tasks. In addition, in the beginning, some applications created one task for one interface, but in fact, an application can set multiple interfaces independently according to the recording and playback rules of each interface, that is, only one task is needed. Before the completion of the cutting in February 2022, most of the research and development uses R2 to compare the data of the Great Wall project. The March 2022 test uses R2 for daily project and requirement testing. Among them, according to the situation of each business line, the real-time DIFF will be selected first for the business with strong requirements on effectiveness, and the offline DIFF can also be selected for the business with low effectiveness. From the data point of view, most of the demand tests are performed using real-time DIFF.

3.2 Business data problems discovered by using R2 in the Great Wall project

Taking the real-time DIFF of promotional and commodity lines connected to R2 as an example, Figure 3.3 shows the number of bugs discovered through R2 before the Great Wall goes online. Among them, a total of 140+ problems were found in the product & promotion, of which 76 problems were found on the B side of the product, and 37 problems were found on the C side. In addition, 27 problems were found on the B-side of the promotion, and 30 problems were found on the C-side of the promotion. The B terminal in the figure refers to the operation management background, and the C terminal refers to the front end of Qixian.

Figure 3.3

In addition, based on the R2 comparison results, it can be found that there are relatively many attributes involved in the product, so R2 will compare every inconsistent field before and after the flow cut. Among them, the fields with inconsistent comparisons on the B side of the product mainly include product status, missing pictures, product description, product name, and product brand. The main reasons for these data inconsistencies are: (1) data synchronization issues; (2) inconsistencies of 0 and null for certain fields; (3) fields are not aligned. The advantage of using R2 is that these inconsistencies can be compared in advance, repaired and optimized, reducing the risk of project launch.

The problem found in the comparison of the B-end of the promotion shown in Figure 3.3 is that the promotion cannot be found after the stream is switched, but the C-end can be hit. The main reason is that the data synchronization is abnormal and the product functions are not aligned. Therefore, systematic data repair and development are required. Align the function with the product to ensure the quality of the launch. The C-side problem focuses on the response results after the user's real-time request, as long as there are fields or values that are different, they will be compared. The problem of promoting the C-side also focuses on the problem of data synchronization. Therefore, R2 plays a very important role in verifying the data synchronization logic.

4. Summary and Suggestions

This article mainly introduces how the omni-channel quality team uses R2 to ensure business quality. Starting from the introduction of R2, introduce step by step the introduction of R2 to omni-channel implementation and revenue. Based on the promotion business of the Great Wall project, it introduces in detail the application of omni-channel access to R2 and the use of R2, as well as a summary of the problems discovered by R2. Finally, based on the experience of using R2 testing in the Great Wall project, here are some suggestions:

To ensure the quality of business.

(1) When using R2 for task setting, you can create a task related to an application interface, and try not to create a task for each interface.

(2) You can use R2 to obtain some input parameters of the interface, precipitate the parameters related to the interface, and improve the automated test cases.

(3) For interface responses with DIFF, DIFF playback tests can be performed directly after R&D and repair without re-executing the task. In addition, for those with high real-time requirements, DeepTest can be used to manually simulate DIFF requests for verification. If the real-time requirements are not high, you can use R2's offline DIFF playback function.

(4) For the complex structure of the returned results, you can set the fields to be compared or the fields to be filtered, and deposit a task template, which can be reused after minor changes in the later stage to improve the efficiency of creating tasks.

Author: JD Retail Yang Yaxiao

Source: JD Cloud Developer Community