[Deliver high quality, high user growth]-User Growth Quality Assurance Methodology | JD Cloud Technical Team

Preface

As the saying goes, "testing is the guardian of quality," but testing alone is not enough. In most cases, testing is like a "mirror" that reflects the face of the system and provides developers with a basis for modifying the code. This process of "looking in the mirror" is the process of quality assessment, or in other words, the testing process is more like " "Measure body temperature", although the temperature can be measured to determine health status, it cannot be used to treat diseases by measuring body temperature. At the same time, the high quality of demand delivery is not only reflected in the results level, such as functionality, performance, reliability, availability, maintainability, security and user experience, but also includes the process level of delivery, such as the high quality of business requirements, product The high quality of the documentation, the high quality of the test code, etc. Therefore, quality assurance should be viewed from a higher dimension and a broader perspective.

Based on the business scenario of C-end user acquisition, this article summarizes the framework, strategies, processes, specifications, methods, tools and practices in the quality assurance process from a comprehensive perspective of quality assurance, and comprehensively elaborates on the values ​​and methodology of quality assurance for user growth. And the connotation we understand, that is, high quality = diversification of quality strategies + standardization of quality processes + standardization of quality activities + platformization of quality tools + normalization of quality operations.

Limited by the limitations of my own cognition, professional ability and summary ability, some of the opinions in the article may be biased or even wrong. At the same time, it does not mean that our level is high enough to preach and resolve doubts here. The greater significance is that we The process of sorting out this article is itself a process of continuous learning, summary, reflection and finding gaps. At the same time, it may bring you some useful information and valuable thinking.

1. User growth quality assurance values

1.1 Responsibilities for quality assurance

As the “Guardian of Quality,” it is critical to clarify the responsibilities of quality assurance. In our opinion, quality assurance responsibilities can be summarized into the following aspects:

• Determine quality standards

On the basis of attributes such as functionality, compatibility, experience, performance, and security, we clarify the quality standards of business requirements. From the perspective of users and businesses, we use the minimum cost to achieve quality requirements that are acceptable to the business and ensure user experience.

• Develop quality plan

Develop quality plans and strategies to clarify test objectives, test scope, test methods and test schedules, as well as arrangements for other quality assurance activities.

• test execution

Conduct various testing activities, including functional testing, performance testing, security testing, etc., to evaluate the quality of products or services, timely discover and promote the resolution of potential problems.

• Manage defect tracking

Track and manage defect reports to ensure issues are resolved promptly and work with the development team on defect remediation and verification.

• Provide suggestions and improvement measures

Based on test results and quality assessments, quality assurance personnel provide suggestions and measures to improve the quality of products or services to ensure continuous improvement of products or services.

• Quality monitoring and insight analysis

Continuously monitor the quality of products or services, conduct statistical and insight analysis, and participate in the process of continuous integration and continuous delivery to ensure the effectiveness and continuous improvement of quality activities.

• Optimize process specifications and training

For common quality issues, safety issues and user experience issues, optimize quality assurance process specifications, provide relevant training and guidance, and share best practices.

1.2 Quality Challenges

In the user growth field of JD Technology-Market and Platform Operations Center, the overall form of the project is to increase the number of new financial APP, new business and new C-end users through activities, thereby further increasing the number of active users of the APP, increasing Baitiao, Cross-conversion of gold bars, small treasury, funds and other businesses. These activities can be roughly divided into three types: major promotion venues, C-side new attraction activities and marketing tools.

Big promotion venues refer to the venue pages we launch during large-scale events such as 618, Double Eleven, and New Year's Day. By displaying different interest points, gameplay, product floors, and recommendation algorithms, we attract users with different identities to convert; C-side The main forms of attracting new customers include launching various new attracting activities in financial markets, retail stores, and outside markets (WeChat, DouKuai, and other apps). At the same time, marketing activities can be launched in combination with festivals, social hot spots, etc. In addition, you can also play games. The method guides users to conduct real-name authentication, complete nine factors, bind cards, open financial services and other business transformations; the "One-Point Purchase" series is a typical new acquisition tool, which achieves business transformation in the form of new customers paying one cent to purchase, including A series of tools and other items. If classified according to the strategy of attracting new users, they can be divided into three types of projects: new-attracting tools, expanding traffic exposure, and improving acceptance efficiency, as shown in the figure below:

 

For the above three types of projects, we have summarized the following characteristics and quality challenges of use-increasing projects:

1.2.1 Focus on one speed

Based on the particularity and timeliness of user growth projects themselves, many projects themselves are a process of continuous trial and error. Therefore, rapid launch, rapid feedback and rapid adjustment are required. This requires testers to quickly identify and verify various scenarios. Online quality problems must be resolved as soon as possible to minimize business losses and user experience damage.

1.2.2 Details determine success or failure

The success of many activities is reflected in the details, which require constant consideration of user psychology. For example, a cashback project must be repeatedly adjusted in terms of page copywriting, style, interaction, benefit points, etc., by dividing different styles of activity pages and creating a paragraph. Manage time to find the most effective way to transform efficiency. At the same time, in order to prevent users from aesthetic fatigue, the copywriting and styles must be changed periodically. During this process, testing needs to accurately identify the processes and function points that require verification and regression, so as to achieve low-cost and accurate testing.

1.2.3 Return to Deep Like the Sea

Many functions of the user growth project affect each other, affecting the whole system. Therefore, when carrying out functional transformation and addition, we need to perform regression testing on historical functions. This regression testing is not only limited to the main process, but also requires attention to more details. Due to the rapid iteration of the project, testers are required to have efficient testing skills and the ability to accurately locate the regression scope to ensure the stability and reliability of the project. Through continuous regression testing, we are able to discover and solve potential problems and improve the quality and user experience of the project.

1.2.4 High experience requirements

Experience issues are also one of the things that need to be focused on in user growth projects. Since most of the projects are directly oriented to C-end users, there are some minor problems, such as loose vocabulary, abnormal jumps, too large/too small fonts, uncentered button positions, unclear prompt information, and too many retention pop-ups. Hot spots that are too large and the close button is not obvious may cause customer complaints, and the friendliness of the user experience is directly related to the efficiency of user growth and business conversion. Therefore, when testing requirements, test case review, and test execution, test students should treat themselves as ordinary users to think about whether the interaction is reasonable and whether there is room for optimization of the experience.

1.2.5 High pressure of capital losses

Most user growth projects will issue various rights and interests to users. Whether they will be defrauded, how to deal with risk control users, how to prevent capital loss risks, how to identify abnormal processes, and how to take precautions are the top priorities that need to be considered. On the other hand, in order to ensure that the operation strategy can be adjusted at any time during the project operation, copywriting, pop-up windows, interest points, crowd strategies, etc. are generally all configured. How to simplify configuration from the system level and prevent configuration errors is also an important issue that needs to be solved. Once the configuration is wrong, it may lead to huge capital losses or a large number of customer complaints.

1.2.6 Difficulty in preparing materials

Since C-side new attraction activities need to be targeted at various types of new accounts (without real names, without 9 elements, without card binding, C-side new, new business, etc.), the activities should be visible and obtainable, and the identity change can be achieved after receiving benefit points. During the testing process, various types of accounts are required to test and verify various normal and abnormal scenarios. The difficulties are mainly reflected in the large demand for test accounts, accounts cannot be reused, the same account cannot complete the full process verification before going online, and real accounts (new on the C-side, new on the business) are required for acceptance after going online, etc. Although testing can be supported internally by adding whitelists to accounts, applying for new internal accounts on the C-side, applying for super accounts, etc., due to the reliance on multiple external systems such as risk control, real-name, marketing, banking, etc., it is difficult to test accounts in most cases Verify the entire process.

1.3 Quality values

Focusing on the user acquisition business field, combined with the business characteristics, quality practices and our understanding of quality in this field, we summarized the following aspects as reflections on the quality values ​​of user growth, and reached a consensus within the team.

1.3.1 National quality assurance

Quality is not the exclusive responsibility of testing students, nor can we achieve high quality just by testing students. Quality requires the participation of all employees, and everyone is the guardian of quality. In our quality practice, there are 7 lines of defense in the entire delivery process to prevent defects from escaping online. Although business/operations, products, R&D, testing and project management have different divisions of labor, they are all indispensable roles in the quality defense line. .

1.3.2 Prevention is better than detection

In the quality assurance process, preventing the occurrence of problems is more important and effective than discovering and correcting problems afterwards. Taking preventive measures before problems occur to eliminate potential quality problems at the source. The cost of prevention is often lower than the cost of defect repair. . In our practice, the main measures to prevent defects include: emphasizing the accuracy, clarity and completeness of requirements documents; strict code review before testing, which can reduce potential problems and errors in the code; early intervention in testing to ensure Products meet needs and fundamentally improve product quality; to promote quality culture, all roles should be aware of the importance of defect prevention.

1.3.3 Quality is free

In order to ensure delivery quality, you should try to invest enough time, resources and energy in product design, development and testing. Although the investment incurred in the short term may increase costs, the additional costs related to quality problems (customer complaints, capital losses, R&D repair costs, etc.) may be higher, and the improvement in quality will ultimately be reflected in stable online quality and improved performance. , delivery efficiency improvement and user experience improvement. So, in a sense, quality is not a cost, quality is not only free, but also generates additional profits.

1.3.4 Quality cannot be measured

Testing itself cannot effectively improve quality, just like a thermometer cannot lower body temperature. Quality built-in is the core of quality assurance. Quality needs to be paid attention to in all aspects including demand analysis, R&D design, development, testing, launch, operation, etc. Improve the quality awareness of the whole people, discover defects as early as possible, implement agile DevOps concepts, tools and methods, establish and continuously optimize the R&D process and online specifications, and establish a full-process quality assurance system through quality pre-positioning and testing right shift. Through these built-in quality methods, the occurrence of defects and problems can be effectively reduced, quality and stability improved, while subsequent testing and repair work can be saved, and R&D costs can be reduced.

1.3.5 Quality is not always better

Excessively high quality levels may lead to resource waste and unnecessary cost increases. Quality should be consistent with the actual needs of the business and usage scenarios. Excessive pursuit of perfect quality may lead to extended development cycles, increased costs and even unnecessary impact on user experience. limitations. In short, quality means meeting the needs, rather than exceeding the needs, taking into account cost, schedule, risk and other factors.

1.3.6 Quality is not intangible

Quality can not only be felt, it can also be measured. In the field of user growth, our commonly used quality measurement indicators include: average number of defects per requirement and number of defects per R&D employee, to measure the density of defects; defect closure rate and defect closure duration, to measure the completion and efficiency of defect resolution; return The roll rate is used to measure the stability and reliability of the requirements to be delivered; the number of online incidents is used to measure the number of defects escaping online, indicating the stability of the online system. Through quality measurement, quality goals can be transformed into specific measurable indicators to better guide and monitor quality improvement.

2. Brief description of user growth quality assurance methodology

2.1 Quality Assurance Methodology

Combining long-term quality practices in the field of user growth and the quality values ​​we recognize, we have sorted out the quality assurance work in the field of growth. Different quality activities need to be taken in the process quality, release quality and online quality assurance processes, and different applications are required. Quality strategies and the use of different quality tools. In order to ensure the smooth progress of quality strategies and quality activities, it is necessary to agree on the quality management process with each role and make the quality concept deeply rooted in people's hearts through continuous quality operations. In summary, quality = quality strategy + quality process + quality activities + quality operations + quality tools. In the field of user growth, high quality = diversification of quality strategies + standardization of quality processes + standardization of quality activities + platformization of quality tools + quality operations Normalization, this is what we consider the quality assurance methodology for user growth.

2.2 Quality assurance system

The department we are in is an R&D team that directly supports business customer acquisition and business growth. We work in the JD Ecosystem, JD Financial APP Center and external private domains (WeChat, DouKuai, Telemarketing and other external APPs) through efficient promotion New tools, event launches, user acquisition tools and other methods are used to attract new users, promote activation and cross-conversion, and continuously expand the scale of users and transactions, thereby promoting the stable growth of C-side businesses such as consumer finance, wealth and payment. Acquiring new users is the most basic task for business growth. It not only helps to maintain competitiveness and promote revenue growth, but also continuously optimizes products and services in the process of acquiring new users to strengthen the influence of JD Financial APP, which is very important to the company. It is said to have important role and significance. Therefore, user growth quality assurance work is crucial.

The above-mentioned quality assurance methodology provides us with the overall goal and direction. We have refined the methodology and broken down the quality strategy, quality process, quality activities, quality operations and quality tools into specific components that can be implemented into the daily work of quality assurance. Matters, combined with the overall quality assurance system of the platform R&D department and the user growth quality assurance methodology, we summarized the panoramic view of the quality assurance system in the field of user growth, as shown in the following figure:

The following will elaborate on the connotation of user growth quality assurance methodology and our practice from five aspects: quality strategy, quality process, quality activities, quality tools and quality operations.

3. Diversification of user growth quality strategies

Most user growth projects are targeted at specific users in the form of activity placements. Various new attraction tools or components need to be comprehensively used to expand traffic exposure, improve acceptance efficiency, and ultimately achieve conversion rates. The main goals include delivery channels and activity scenarios. , applicable groups, front-end interactions, user operation paths and interest points are diverse, and the corresponding quality requirements and testing processes are also quite different. Different quality strategies need to be adopted according to different demand types and business scenarios. In user growth projects, the following quality strategies are widely used.

3.1 In terms of testing concepts, comprehensively apply the practices of agile testing and traditional testing

As a part of the agile R&D model, agile testing focuses on iterative testing and incremental delivery. The system is gradually built and improved in continuous iterations. It does not emphasize the comprehensiveness and test coverage of test cases, and even exploratory testing can be used to replace completeness. test cases, with the main goals of supporting rapid iteration and collecting user feedback. For example, in a project team with high team tacit understanding and stable personnel, the team adopts agile testing methods, conducts testing activities according to the rhythm, quickly verifies, and goes online quickly. Sometimes, in order to ensure rapid delivery, the team can tolerate the temporary lack of functions, performance and experience. In exchange for rapid delivery and customer verification of main functions, and rapid iteration and optimization in subsequent versions. The agile R&D model improves collaboration efficiency and R&D productivity, but it is not necessarily suitable for all scenarios. The traditional waterfall delivery model also has its unique advantages.

Traditional testing pays more attention to complete test plans and detailed test cases. It is suitable for projects with high requirements on stability, functional integrity, experience or performance. Through a complete test verification plan, complete test coverage is ensured, which is very suitable for functions. For project teams that are extremely important or provide basic capabilities for multiple parties to call upon, fast-paced iterations and frequent launches are not optimal.

In short, there is no difference between agile testing and traditional testing. It needs to be adapted to the testing scenario. For the same project, different strategies must be adopted at different stages. For example, one-cent new tools include one-cent cashback/voucher, one-point recharge, one-minute cash, one-point lottery and other new tools. In the early stage, in order to quickly verify, the agile R&D model and agile testing mechanism are adopted. , quickly launch functions and operational strategies, and put them into different channels to quickly verify the effects. Some small experience problems and functional problems that do not affect the main process can be optimized in subsequent versions. However, when the project matures, the project team A large number of members are transferred to other projects, and there are many new people or temporary support personnel. At this time, rapid iteration is emphasized, and a large number of modifications to the underlying functions without complete testing will definitely have a huge impact on the online quality. At this time, it is very necessary to develop step by step, formulate a complete test plan, and prepare complete regression use cases. At this time, slowness and stability may be the first priority.

3.2 In terms of test execution, a combination of manual testing and automated testing is used

Manual testing can discover some unpredictable problems based on the tester's experience and intuition. For example, in a configuration system for operations, testers test the response and interaction of the user interface by manually clicking and typing. Automated testing can improve testing efficiency and consistency. Testers use automated testing tools, write test cases and conduct automated testing to ensure the stability and correctness of the core functions of the software.

Manual testing is suitable for testing scenarios that require strict details and require more human factors. Testers can get started quickly, have fewer external dependencies and can flexibly use various testing strategies. Generally, manual testing can find more defects than automated testing. . Automated testing is suitable for test scenarios that require a lot of repeatability, stability or large amounts of data. It is like an assembly line and can be executed quickly. However, implementing automated test cases itself requires a lot of time and effort, and due to the fragility of automated testing, The test cases that have been developed must also be continuously updated as the objects under test change, and you also need to pay the cost of maintaining the test cases. In the testing in the field of user growth, manual testing is mainly used for temporary activities, new projects and projects with more complex front-end interactions; for more mature new acquisition tools, operation terminal functions and test regressions, the focus is on automated testing. Automation use cases need to be continuously accumulated and continuously updated and maintained.

3.3 In terms of test cases, use script testing and exploratory testing comprehensively

Script testing is performed by writing test scripts and can cover most common scenarios. It is used to verify various functions by writing a series of test scripts (including test codes, test cases with text descriptions, scripts for operating databases and caches, etc.) Module correctness and reliability. Exploratory testing gives full play to the subjective initiative of testers and finds unknown test scenarios through continuous trial and exploration. No or less test scripts are used during the testing process.

There is a synthetic gem mini-game in the Jingxiaofu GO applet. During the test process, test students combined the characteristics of synthetic games and constantly tried different operations and game paths to discover potential problems and optimization points. Script testing is suitable for covering known scenarios, while exploratory testing, as an important practice in agile testing, is suitable for exploring unknown scenarios and new functions. Proper use of exploratory testing can significantly reduce testing costs. In our quality practice of user growth, we have always encouraged everyone to actively try different testing strategies and methods. Whether it is local exploratory testing, global exploratory testing or mixed exploratory testing, they all have their own continuous exploration and practice in different testing scenarios. practical significance.

3.4 In terms of test scope, comprehensively utilize the practices of test left shift and test right shift

The main goal of shifting testing left is to put quality activities in the front and effectively reduce the cost of later problem repair through various defect prevention work. In the quality practice of user growth, in addition to proactively identifying business process breakpoints, abnormal processes, branch processes, test data and test material requirements, acceptance criteria and other common issues during demand review, we have pre-empted test cases. Through the sorting and review of test cases, we can provide more input to R&D design and code writing, identify problems in the business process and demand delivery process in advance, and communicate with products and development in a timely manner. Shifting testing to the right refers to extending quality control to after the system goes online and during daily operation and maintenance. Through daily manual inspections, automated inspections, experience management, etc., problems with user experience can be discovered in a timely manner. In summary, testing left and right is just a visual expression. The core is that quality assurance should not only focus on the test execution stage, but should run through all stages of R&D quality management and the entire life cycle of demand delivery.

3.5 In terms of testing strategies, comprehensively apply strategies such as pyramid, ice cream, and olive models

Pyramid model, ice cream model and olive model are commonly used test strategy models. They have different emphases in test coverage and test priority. The pyramid model divides the test levels based on the priority and number of test cases. The bottom of the pyramid is a large number of unit tests or unit component tests, followed by a small number of integration tests, and then a smaller number of system tests and user interface tests. The goal of the pyramid model is to focus more testing on lower-level testing, discover and solve problems through more fine-grained test coverage, and maximize testing efficiency and accuracy.

The ice cream model emphasizes functional and non-functional testing and focuses on user experience. This model is particularly suitable for user-driven testing, such as user interface testing, user acceptance testing, etc. The olive model combines the advantages of the pyramid model and the ice cream model, taking into account test coverage and user requirement verification. The pyramid model is suitable for small projects with rapid development or projects with complex server-side logic. The ice cream model is suitable for projects that place great emphasis on user experience, while the olive model is more suitable for medium-sized projects. The above three types of test strategy models are just some common guidance methods. Appropriate test strategies should be selected based on specific projects and testing needs. Sometimes a combination of multiple models may be needed.

Compared with other business scenarios, the user growth field mainly interacts directly with C-end users and business operators, has fast iteration speed and high user experience requirements. Therefore, the ice cream model is widely used. For all requirements, we will arrange product verification before going online and operational acceptance after going online, to verify the system's functions and user experience from the three perspectives of the demand proposer, product solution designer, and end user.

3.6 In terms of test collaboration, comprehensively consider the balance between local optimality and global optimality

In the collaboration between testing and other roles, not only the independence and correctness of the test unit are considered, but also the efficiency and quality of the entire requirements development and delivery process. This comprehensive consideration helps optimize teamwork and improve the overall quality and delivery speed of the product. For example, in an agile development team, testers collaborate with developers, product owners, and other roles. From the perspective of local optimality, testers may only focus on checking whether local functions comply with requirements and specifications, while ignoring the needs and expectations of other roles. However, from a global optimal perspective, testers should participate in requirements discussions and work closely with other roles to ensure the accuracy and comprehensiveness of requirements.

This avoids changes and delays in subsequent development processes and increases the productivity of the entire team. In the same way, during the delivery process, we advocate regular reviews, development according to rhythm, scheduling based on objective workload evaluation, smoke admission and other practices to prevent the local optimality of a certain stage or a certain role from leading to global low performance. inefficiency or low quality. However, in real work, sometimes the global optimum has to be sacrificed for the local optimum. This has a lot to do with the stage of the project, the organizational environment, internal and external business pressures and other factors. Therefore, it is very important to comprehensively consider local optimality and global optimality in the collaboration of testing and other roles, because this helps promote effective communication and collaboration in the team, and improves overall work efficiency and product quality. By considering both local and global needs and expectations, test students can play a greater role in the development and delivery process.

To sum up, in the process of quality assurance for user growth, it is necessary to comprehensively apply and combine various quality strategies according to different business scenarios, project requirements and organizational environments, and adopt different test plans and testing methods to improve delivery quality and reduce potential failures. Problems occur and enhance user experience. What needs to be emphasized is that in the practice of quality assurance for user growth, we will not use the above strategies rigidly and mechanically, but apply them to daily quality work to ensure that everything is smooth and silent.

4. Standardization of user growth quality process

Quality does not occur during the testing phase, nor does it end during the testing phase. Quality runs through every aspect of the entire product life cycle, including solution design, development, testing, release, operation and offline. In the agile development model, testers will start testing as early as possible, including timely review of requirements, R&D designs and test cases. More importantly, they can provide timely and continuous feedback on the quality of products/projects/requirements, which can be summarized as the following figure :

In view of the characteristics of product delivery for user growth, the testing team needs to complete the work "fast and well", and needs to implement quality assurance, cost reduction and efficiency improvement throughout all stages of demand delivery, so we are at every stage. Developed corresponding quality process standardization specifications. We summarize testing activities into the stages of "requirements analysis", "use case analysis and design", "test execution stage" and "online quality monitoring stage". Next, we describe in detail the specific working methods at each stage.

For example, in the demand stage, we formulate requirements access specifications, including demand document content display, front-end page interaction, etc., to avoid unclear requirements during the development process, which will lead to product verification, R&D, modification, and testing in the testing stage. Verification; in the use case design stage, the test case review is pre-reviewed. From the functional level, the product demand details can be supplemented. From the system implementation level, the affected function points and regression test points can be listed according to the R&D implementation method to improve the use cases. Effectiveness; when products are released, a grayscale release plan will be formulated based on demand characteristics; during online operations, regular inspections of online functions and overall dynamic data monitoring of the system will be carried out.

4.1 Requirements analysis: Identify business value and commercial value

Dave Hendricksen pointed out in his book "The 12 Disciplines of a Software Architect": "When system architects consider the true value of software architecture, they should not only focus on the technology of system construction, but also on customer value and business value - you You have a deep understanding of what kind of problems you can really help customers solve and how you can help the company make money." Even if the product uses the most advanced development technology, if it cannot meet the pain points of users and solve business demands, such a product is useless and cannot be called a successful product. During the project startup phase, the in-depth participation of testers is crucial and helps to identify business value and commercial value, which is reflected in the following aspects:

• Insight into the business: Insight into the business is the basis for quality assurance. Through in-depth understanding of the business, testers can better implement testing strategies, create accurate test cases, and cover important business scenarios and processes to ensure that business requirements are delivered in production and research. correctness and completeness. It can be said that without in-depth understanding and insight into the business field, quality assurance will become water without a source and a tree without roots.
• Understand the value of requirements: Understand the goals and values ​​of requirements, and at the same time, based on communication with business and product managers, understand the business meaning behind the requirements and the business problems that are solved, not just the functional description in the requirements document.
• Understand the required functions: Deeply understand the user's usage scenarios, understand the business operation process, as well as the functions corresponding to the requirements and their position in the business process. Our systems are usually of high complexity and may involve multiple modules in one project, or the functional collaboration of multiple projects and departments. Therefore, all things need to be considered when cascading processes.
• Analyze user experience: In-depth understanding of product system, architecture and interactive relationships. From the perspective of business processes, ensure the rationality, fluency and completeness of the business processes and ensure that users have no doubts during the operation. Think about usage scenarios and operation processes from the user's perspective, allowing users to operate less and think less.

4.2 System analysis: Determine whether the functions to be delivered can meet business needs

When designing requirements verification points, you need to consider "visible requirements" and "invisible requirements". "Visible requirements" are the functions that need to be implemented as described in the product manager's requirements document. "Invisible requirements" refer to the technologies used by R&D students in the code implementation process. These technologies may require some test cases for verification. Therefore, the input of our requirements analysis not only includes product requirements, but also must consider the specific implementation of the code. An in-depth understanding of the detailed design of the R&D before testing begins will help testers formulate appropriate testing strategies. During the requirements delivery process, you can gain an in-depth understanding of R&D design through the following methods:

• Understand the implementation process of R&D design, including system architecture design, module division, and technology and data interaction of each module. For example, in terms of data storage, you need to have a detailed understanding of the characteristics of different data storage and corresponding implementation solutions, such as relational databases, NoSQL databases, Redis, ES, etc., in order to evaluate their performance and availability; at the data interaction level, you need to have an in-depth understanding of the data in the system. Interaction methods, including synchronous and asynchronous calling methods, data format conversion, encryption and compression of data transmission, etc.; Understand various exception types and corresponding processing methods of the system, including error code definitions, exception logging, alarm mechanisms and exceptions Rollback processing, etc. to ensure system reliability and maintainability.
• In terms of system configuration, it includes the format of configuration files, classification and naming rules of configuration items, dynamic update and preloading mechanism of configuration, etc. to ensure the flexibility and scalability of configuration; in terms of scheduling tasks, it includes task scheduling frequency and scheduling strategy. , failure retry mechanism, job priority management, etc. to ensure reliable execution and efficient scheduling of tasks.
• Understand dependencies and interactions with external systems. When attracting new users in the WeChat domain, a large number of demands involve interaction with enterprise WeChat and mini programs. Enterprise WeChat actively and passively add WeChat, send group messages, and operate communities, etc., which require a detailed understanding of WeChat, user growth activities, mini programs, and Tencent side systems. Interaction, such as interface definition, protocol specification, data exchange format and data verification, callback method, retry mechanism, etc.
• Code change points and calling links are sorted out. The user growth project has been in a rapid iteration process, and the function points and calling relationships involved in the project are relatively complex. For R&D implementation, it is necessary to further sort out the calls and impacts of each code change point on other modules, and analyze the potential problems it may cause to ensure the stability of each change point and affected function point.

4.3 Test execution: Ensure that the functions to be delivered are consistent with the explicit and implicit requirements of the product

After the above requirements analysis and system analysis, we can design test cases based on these inputs. In summary, test cases can be designed from three aspects: business function analysis, system implementation analysis, and delineation of the impact scope of changes in system implementation. After the test case review is completed, the test is executed. Effective test execution can ensure the correctness of software functions, improve software quality, and reduce software maintenance costs. Results and problem feedback during the testing process can help the development team fix problems in a timely manner and improve the software development process, thereby improving the overall delivery quality. and development efficiency, therefore, test execution is a key link to ensure product quality.

In addition to the testers' own quality activities and behaviors, the test execution stage is also the stage where communication with business/operations, products, R&D, etc. is most frequent. Therefore, reasonable and standardized test process constraints and rapid attribution of quality issues are the guarantee of high-quality delivery. . Dividing the testing process into multiple nodes and sharing them can make the division of responsibilities clear, which is helpful for relevant personnel to analyze the causes of quality stuck points and control progress. During the testing phase, functional testing and non-functional testing need to be completed. Functional testing is an explicit verification of product quality, and non-functional testing is an in-depth inspection of product quality.

• Functional testing: The goal of functional testing is to verify whether the software system implements the functional requirements defined in the product requirements document. The steps for functional testing are generally: designing test cases, preparing test environments, executing test cases, verifying test results, defect tracking and review Implementation, defect repair and regression testing, sending test reports and summaries, etc. In order to adapt to the requirements of agile delivery, we adhere to the principle of shifting testing to the left and move some testing steps forward, such as test case design & review before coding, test environment construction and Material preparation is completed before R&D testing. Adhering to the principle of shifting testing to the left can help the entire team discover problems earlier in the agile development process and reduce the cost and risk of later repairs. At the same time, through the pre-testing step, it can promote close collaboration between testing and development and improve the quality of requirements delivery. and efficiency.
• Non-functional testing: The functional part of the business requirements has been verified through functional testing, but the quality testing has only completed the first stage. The second stage is the verification of software performance and potential risk points to ensure that the requirements are foolproof after delivery. Non-functional testing mainly tests the reliability, usability, performance, security, compatibility and other aspects of the belt delivery requirements. Non-functional testing usually requires the use of specialized testing tools and technologies, and the design of appropriate test scenarios and test conditions for test verification. For example, the twice-annual major promotion stress tests use Forcebot, Titan and other stress testing tools for performance testing.

In addition, some work is difficult to distinguish between functional testing and non-functional testing, but it is also very important quality assurance work, such as verification of buried points and data accuracy. Main tasks include:

• demand stage

In the demand stage, promote the review of buried requirements, identify the rationality of buried design plans, and promote the improvement of the quality of buried requirements.

• testing phase

Including buried point test strategy design, manual buried point verification, automated buried point verification and other work.

• Buried point online stage

Test classmates perform regression verification, and promote product and quantification personnel to conduct acceptance inspection.

• Data application stage

Verify the consistency and accuracy of data warehouse data and application layer data through the data measurement platform to improve the comparison scope and efficiency; propose optimization suggestions for the analysis platform from the user perspective to improve the user experience of the data application platform.

4.4 Online Release: Perform pre-launch check and post-launch grayscale release

When the requirements development phase came to an end, the team's work also entered the critical online release phase. In order to ensure that the product functions that have completed quality testing are put online normally, before the requirements are put online. The user growth quality team, in collaboration with other roles, usually needs to do the following work during the go-live stage:

• Product acceptance testing: During the business acceptance testing process, the testing team will simulate real business scenarios, test the operation of the functions to be delivered in different business processes in the test environment or pre-release environment, and compare and verify with the product manager and business parties , to ensure that the functions of the software can operate normally.
• Checklist execution before launch: We have sorted out the checkpoints and concerns at each stage from requirements review to launch verification. Some are optional actions and some are required actions to avoid omissions in the delivery process or lack of pre-communication. Leading to rework or online problems. Checklist check items are divided by role and divided into four parts: operational checkpoints, product checkpoints, R&D checkpoints and test checkpoints. The R&D checkpoints mainly include code merging, whether compilation and packaging are completed, whether the database upgrade is successful, etc. Test checkpoints mainly include whether the test environment is complete, whether the test cases are completely executed, and bug fixes, etc. The testing team needs to carefully check this Checklist to ensure that each checkpoint has been completed to ensure the stability and reliability of the software before going online.
• Closure time control: After the closure time, except for online bugs, the development team will no longer accept new functional requirements and modification requests. The testing team needs to control the closing time and ensure that all testing work has been completed before closing. Controlling the sealing time is very important for the software launch process. It can ensure the stability and quality of the software and reduce problems and risks after it is launched. Currently, projects related to the C-side of Microelectronics will implement the agreement to close the board before 12 o'clock on the day of launch and start to go online at 18 o'clock in the evening.
• Version grayscale release: Before the requirements go online, version grayscale release is usually carried out, that is, new functions are gradually released to some users to verify the stability and reliability of the software in the real environment. The test team participates in communicating the grayscale release plan, collaborates with products and operations to determine the grayscale population, grayscale proportions, volumetric plans and emergency plans, and collects user feedback and problem reports.

Generally speaking, in the online stage of requirements, the Yongzhu test team needs a series of tasks such as business & product acceptance testing, pre-launch Checklist execution and board closure time control, and version grayscale release. Completion of these tasks can ensure that the software can run stably and meet user needs after it goes online. The testing team needs to work closely with the operations, product and R&D teams to ensure the smooth progress of all work and ensure high quality and on-time delivery.

4.5 Put into production and operation: collect user feedback and accumulate experience and lessons

After the requirements are put online, the quality improvement team still needs to coordinate with all parties to carry out a series of quality assurance work, mainly including:

• Monitor and collect feedback: Monitor the operation of the software, learn in time whether there are any abnormalities or customer complaints, and ensure that online problems can be repaired as soon as possible. The quality team also needs to actively collect user feedback, provide more input for business planning and product optimization plans, and continuously improve the user experience.
• Bug tracking and repair: Due to time issues, some defects may need to be fixed gradually after going online. The quality team collaborates with the business and products to evaluate the priority and severity of the problem, and coordinates the development team to fix it in a timely manner.
• Performance monitoring and optimization: The quality team needs to continuously conduct performance monitoring, including evaluating the software's response time, concurrent processing capabilities, and resource usage. If performance issues are discovered, work with the development team to optimize performance and user experience.
• Automated testing and continuous integration: After going online, the quality team can further promote automated online testing and continuous integration. Through automated inspections, online problems are discovered in a timely manner and notified to the project team as soon as possible. At the same time, continuous integration can help the testing team fix problems faster.
• Precipitation of product function documents: After going online, the quality team sorted out the problems and difficulties encountered in the entire testing process, and organized the business understanding, testing strategies, testing methods, experiences and lessons, etc. into words. On the one hand, it helped self-summarization and Reflection, on the other hand, helps share it with other students to achieve common improvement.

In short, quality process standardization plays an important role in ensuring product and demand quality. It ensures that products can meet the expectations of customers and business parties, and provide reliable performance and lasting value. Through standardized quality processes, differences within the team can be quickly aligned. The quality assurance work of test students with different ranks and abilities ensures that every step is carefully reviewed, tested and verified to minimize the chance of defects escaping online. At the same time, through continuous summary and precipitation, the quality process is continuously optimized and improved. .

5. Standardization of user growth quality activities

The so-called quality activities refer to a series of activities and processes carried out to ensure the quality of products or services, aiming to prevent and detect potential quality problems and take corresponding measures to correct and improve them. The operation of user growth requires constantly trying different user growth strategies and quickly adjusting based on user feedback and data feedback. At the same time, it is necessary to quickly follow market hot spots and quickly iterate product functions. In order to meet fast delivery without sacrificing product quality, we have developed a quality-increasing access control system to provide quality access and exit at each stage of demand delivery through standardized quality activities, the so-called seven lines of defense.

5.1 The first line of defense: testing the front end

TDD (Test-Driven Development) is an important practice of agile testing. It emphasizes writing test code before writing code to drive the improvement of code quality and functional coverage. Combined with the current status of quality assurance in the platform R&D department, most of the test cases are in the form of text descriptions written in Therefore, in the first stage, we prioritized the implementation of test cases, that is, the writing and review of test cases are preceded by design review or code development, and the functional requirements, performance requirements, exception processes, data requirements and Acceptance standards, and make up for the functional points and process deficiencies that may be missed in the requirements review process, prevent defects in advance, and reduce rework and repair costs in the later testing stage. Through the pilot projects of multiple projects in the fields of user growth, microelectronics, etc., all parties have given positive feedback, and the scope of the pilot is currently being expanded, with the goal of realizing 80% of requirements in advance of use cases.

5.2 The second line of defense: unit testing

Unit testing is an independent test of the smallest testable unit in the software (ie, functions, methods, classes, etc.) in the code. Its main purpose is to verify that each unit works correctly as expected. Unit testing has several benefits:

• Improve code quality: By writing unit tests, developers can verify that each unit behaves as expected, which can help uncover potential bugs, edge cases, and anomalous behavior.
• Ensure independence between modules: Unit testing requires each unit to be tested independently, helping to build more flexible, scalable and maintainable code.
• Support refactoring and code reuse: It can help developers verify whether the refactored code can still work correctly and ensure that reused components behave as expected in the new environment.
• Reduce debugging time: Unit testing can quickly identify problems, narrow the scope of debugging, and speed up problem troubleshooting.
• Build confidence and documentation: By writing comprehensive unit tests, developers can build confidence in the behavior of their code and, when the code changes, can quickly run tests to verify that the code still works correctly.

In short, unit testing is an effective software testing method. It is implemented and executed by developers' coding, which fully embodies the concept of national quality assurance. In the user growth project, R&D focused more on unit testing and wrote a large amount of unit test code while coding. In particular, the user growth R&D team connected to ChatGPT and joined forces with other departments of the group to form a JoyCoder joint project team. Through continuous iterative optimization, it is now possible to quickly and automatically generate more standardized unit test code, which can greatly reduce the workload of unit testing.

5.3 The third line of defense: smoke testing

Smoke testing plays a role in early screening of problems and preliminary assessment of product quality in product quality assurance. It is an important part of ensuring product quality. Qualified smoke testing can quickly screen problems, help the team optimize resources and work allocation, and achieve a preliminary assessment of product quality, which can promote the improvement of team delivery efficiency. In the practice of user growth quality assurance, we generally verify whether the system is suitable for more in-depth testing in the initial stage by running a set of basic test cases for key functions and core processes. We generally use smoke demonstrations. The R&D team believes that it has the ability to improve After testing the conditions, invite test students to demonstrate and review the smoke use case on site. In our practice, about 30% of the total use cases are generally marked as smoking use cases, which are generally verification points for the main process and core functions. The proportion of smoking use cases may vary greatly for different requirements, which is related to the difficulty of the requirement, the number of core main processes involved, etc. Under normal circumstances, it is easy for R&D and testing to reach a consensus on the content and proportion of smoking use cases.

5.4 The fourth line of defense: test execution

In the product delivery process, the execution of tests is the fourth line of defense for product quality assurance and one of the most critical steps to ensure software quality. Through effective test execution, product defects can be discovered as early as possible. The types of defects include but are not limited to: functional problems, user experience problems, performance problems, security vulnerabilities, buried point specifications, compatibility, risk control and anti-brushing, etc. The test execution phase is the longest period of work for test students, and it is also the testing work content that other roles are most familiar with. Usually the demand defects discovered at this stage can reach more than 95%. Under normal circumstances, the workload in the test execution stage accounts for 30% to 50% of the overall R&D work. Of course, the proportion of testing workload may vary depending on different requirements. Larger, especially the proportion of regression testing and the proportion of automated testing in regression testing, will directly affect the workload and duration of the test execution phase.

5.5 Fifth Line of Defense: Product Verification

Product verification is the fifth line of defense to ensure software quality, including UAT, UI walkthrough and experience acceptance. Before the requirements are ready to go online, we will invite the product manager to verify the functions to be delivered in the pre-release environment or test environment. At this time, the testers and product managers will participate in the system verification of the product, and the test students will demonstrate the main process or the product manager will do it independently Verify functionality, performance, and user experience meet original requirements and expectations, as well as verify operational configuration for issues. The results of product verification are divided into two situations: pass and fail.

For those that pass, we can start working on the final release and delivery. For those that fail, we will provide feedback to the development team as soon as possible so that the problem can be repaired and optimized in a timely manner. During the product acceptance stage, based on product design and user perspectives, product managers can put forward various views and opinions to further improve the product. This kind of diversified feedback and opinions can help the team identify and solve potential problems before going online. Although it is already in the late stage of demand delivery, since the system has not yet been launched to customers, there is still a certain amount of time to fix the problems, so as to avoid them as much as possible. Problems escape online to generate customer complaints.

In addition, if there are more front-end interaction requirements, it is necessary to invite UI designers to conduct UI walkthroughs and user experience colleagues to conduct experience acceptance after product verification. As an acceptance of user operations and user experience before going online, if the acceptance fails due to defects in the experience, the user experience colleagues have the right to decide to postpone the launch until the optimization is completed or all parties reach a consensus on the experience issues, then the user experience can be launched first. And complete optimization before wide-scale launch.

5.6 The sixth line of defense: operational acceptance

Operational acceptance is mainly to invite operation students to conduct final acceptance online after the requirements are online. Operational students stand from the business and user perspectives to verify whether the functions to be delivered are consistent with the original expectations. The operation acceptance stage is the last step before the functions are released to customers. The line of defense is based on deep insights into users, keen intuition, and in-depth research on similar functions on the market. At this stage, operation students can often discover some problems or defects that are easily overlooked. At the same time, more importantly, you can verify whether there are any problems with the backend configuration, whether the budget is sufficient, determine the diversion ratio of new and old functions, whether defects are within the tolerance range, whether it is necessary to report to customer service, and determine the operational strategy and operational rhythm after launch. and subsequent product iteration planning. At this stage, there may occasionally be disagreements between operational opinions and product opinions, as well as R&D and testing opinions. Therefore, this stage is also an important stage for mutual persuasion and alignment of understanding.

5.7 The seventh line of defense: disaster recovery drills

With the widespread popularity of business development, microservice architecture, distributed architecture and virtualized container technology, the complexity of software architecture is constantly increasing, and the uncertainty caused by the dependence between services has also increased exponentially. In this way In the service call network, normal or abnormal changes in any link may have a butterfly effect-like impact on other services. As users grow, online marketing activities, new tools, and public components continue to increase, the overall link growth and data flow are complex, and the availability and stability of the entire system are increasingly challenged, so it is very necessary to proactively find out Vulnerable links in the system are then reinforced and prevented in a targeted manner to avoid serious consequences when failures occur, further improve the high availability of the business system, and improve the emergency support capabilities of the business system.

In recent years, several large-scale failures have occurred at home and abroad, resulting in long-term interruption of services to a large number of users, which has had a huge negative impact. In order to effectively reduce the impact of internal and external environmental failures on the system, we simulate various types of failures in daily work to test the impact on the system and the risk response capabilities of the research and development team. We conducted two types of content in the field of user growth. Disaster drill:

• One is Chaos Engineering at the application level

Chaos drills are a practical method of testing and improving system reliability by intentionally introducing system randomness, instability, and failures. It is designed to help organizations identify and resolve potential system flaws and performance issues to reduce system failures and improve System fault tolerance. The key concept of chaos drills is "find faults by introducing faults." By introducing unstable factors and failure scenarios in a controlled manner, such as shutting down a service, simulating network delays, causing hardware failures, etc., chaos drills can verify the system's resilience, fault tolerance, and recovery capabilities. It can help us discover hidden system weaknesses, identify performance bottlenecks and independent failure points, and provide opportunities to improve system stability and reliability.

• The second is the high availability of data storage and the network outage disaster recovery drill of the computer room network.

Scenarios for the drill include operator network disconnection, JD Cloud computer room disconnection, storage device disconnection, network traffic jitter, network traffic packet loss, etc. The impact may be wider, so the drill content and emergency plans need to be sorted out in advance, including Sort out drill SOPs according to different scenarios, set drill templates according to SOPs, evaluate whether the system meets drill requirements according to the templates, upgrade and transform the system according to drill requirements, and design drill processes and checklists according to the drill templates to ensure that the online system will not be affected by the drills. By sorting out the drill process, drill content, risk matters, and response plans, we can ensure that in the event of a similar basic failure or network or database switching, SOP operations will be executed in an orderly manner, and the system will be in a risk-controllable state. So far, the database and cache switching drills for three core applications in the field of user growth, including prize registration, prize issuance, and funding components, have been completed and have achieved the expected results.

This chapter introduces the seven lines of defense set up to ensure delivery quality while delivering products quickly in the field of user growth. Each line of defense is like an access control. Only when access requirements are met can one enter the next stage, thereby standardizing each stage. quality activities and serve as the execution standard for the entire quality assurance process. It should be pointed out that in actual quality practice, the above quality activities are not performed metaphysically and simply and crudely. We will make flexible adjustments or tailoring to a certain extent based on the actual situation of products and business needs to achieve a balance between quality and efficiency. A dynamic, moderate balance.

6. Platformization of user growth quality tools

According to the description in Chapter 1, in the quality assurance process of user growth, there are many challenges in delivery quality and efficiency. These challenges cannot be solved simply by relying on quality strategies, methods, process specifications and endless overtime. Quality tools are here plays an important role in the process. There are many types of quality tools, including automated testing tools, performance testing tools, security testing tools, data construction tools, etc. We sorted out the testing tools commonly used by the testing team and found that a large number of internal and external tools are used in the testing process. The following are the more commonly used tools:

The above tools are independent, and most of them are designed to solve single point problems in the quality assurance process. What we have always advocated is the agile DevOps concept, which emphasizes seamless collaboration and continuous collaboration between various roles in each team. Delivery and automation, pursuing faster delivery speed, higher quality and better user experience in the demand delivery process. In our practice, we generally use demand management, pipelines, defect management, branch management, release tools, quality dashboards, pipelines, etc. based on the Xingyun base as basic quality tools.

At the same time, based on the characteristics of our quality department and years of accumulation, we used the self-developed Jingdu platform to realize the online quality process management, and connected to Xingyun, realizing the demand management module and defect management with Xingyun The opening up of modules includes quality process management, interface/UI automation, H5 performance testing, designated scene traffic playback, big promotion performance testing workbench and other main functional modules. In the Jingdu platform, we can add business systems through "Basic Data Configuration" , database information and interface information, etc., through the "Data Factory" to help test students' intelligent number creation, through "Automated Testing" to achieve the correctness of online inspection interfaces and UI pages, and through "Automated Tasks" scheduled regression testing processes to output detailed The test report can also be used to help solve problems such as knowledge base alignment of test classmates through the "intelligent assistant". Through platform tools such as Xingyun and Jingdu, the entire quality assurance process has been online and digitalized, reducing collaboration costs and providing a basic guarantee for high-quality delivery.

7. Normalization of user growth and quality operations

Quality operations focus on the all-round control and continuous improvement of quality, integrating quality processes, specifications, systems, standards, templates, best practices and quality awareness throughout the entire department, various roles and the entire life cycle of demand delivery. Through measures such as quality governance, quality measurement, quality culture construction and promotion of continuous improvement, the quality concept will be deeply rooted in the hearts of the people, quality activities will be implemented in place, and through the collaboration of various roles, we will ensure that the delivery quality is stable at a reasonable level and space. This series of Initiatives become routine in the quality assurance process. Quality operations are not just the responsibility of a single functional department or individual employees, but the common responsibility of the entire organization. Quality operations can help us achieve continuous quality improvement and optimization. Normalized quality operations include the following aspects:

7.1 Quality Management

By establishing a series of mechanisms, processes and standards to ensure product quality meets specifications and business expectations, and to ensure delivery quality and efficiency, it mainly includes the following contents:

7.1.1 Technical quality management

Improve the quality of test submission and online stability by strictly controlling smoke quality, pre-setting test cases, complete R&D design documents, standardized code reviews, improving the quality of test case reviews, regular review of defects, and supplementing automated inspection use cases. sex.

7.1.2 Data quality management

In view of the pain points such as irregular tracking and difficulty in obtaining numbers, we teamed up with Singularity Research and Quantitative Analysts and other teams to conduct special management of the tracking problem in the market and platform operation center. It mainly includes several aspects such as upgrading the tracking specifications, transforming the tracking reporting platform, piloting the high-code tracking process, and improving the capabilities of the tracking platform. To ensure the accuracy of data collected from buried points falling into the data warehouse, promote the timeliness and accuracy of display of business data on the analysis and insight platform, and lower the threshold for non-data analysts to use the role.

7.1.3 User experience management

User experience is crucial in product and service development for user growth. By focusing on user needs, providing easy-to-use interfaces, designing elegant interactions, and providing pleasant usage experiences, user satisfaction, competitive advantage, and brand image can be enhanced. In order to maximize user value and improve the efficiency of acquiring new users. In response to user experience issues, experience governance is mainly promoted from the following levels.

• Classifying experience issues and establishing experience projects

◦Classification of experience issues: Subdivide experience issues into different categories, such as event experience, new tool experience, operational experience, big promotion experience, etc. This allows for more targeted problem analysis and resolution.

◦ Establish a special experience project: For each category of experience problems, establish a corresponding special experience team or group, and collaborate with various roles to promote monitoring, analysis, and optimization of related experience problems, thereby improving the efficiency and quality of problem management.

• Customer service inbound analysis

Screen user entry problems for key projects, classify and analyze the problems, and give optimization suggestions, such as improving product functions, optimizing interface design, enhancing user guidance, etc.

• Access, detection and monitoring of experience issues

◦ Requirements review and test case review: During the review stage, experience BP and user research students from the customer service department are invited to participate in the requirements review and test case review to control user experience issues and adjudicate controversial experience issues.

◦Front-end test access and acceptance specifications: To address issues such as inconsistent front-end styles and inconsistent specifications, establish standards for front-end test access and acceptance, and review the styles and functions of front-end components such as buttons, input boxes, drop-down boxes, forms, and menus. Unify and standardize it as one of the standards for the entrance test phase of front-end testing.

◦Unification of detection and monitoring indicators: Establish a complete set of performance experience indicator systems to uniformly collect and monitor key indicators, such as loading time, response speed, page stability, etc., to discover experience problems in a timely manner and carry out targeted optimization.

◦In-end H5 performance detection tool: Introduce in-end H5 performance detection tool to monitor and analyze the performance of H5 pages. Through regular testing, potential performance issues can be discovered and repaired to improve user access experience.

◦Production performance scheduled inspection tool: Use scheduled inspection tools to monitor the performance of the production environment and promptly discover and solve system performance bottlenecks and problems. This can ensure system stability and a good user experience.

• Defect removal

◦ Set aside a certain amount of time for defect elimination activities before a major project goes online. Project team members will comprehensively check for possible experience problems and defects in the system and repair them in a timely manner. Through this kind of defect cleaning activity, most functional defects and experience problems can be eliminated before the function is introduced to customers.

Through the above measures, product experience issues can be more comprehensively managed and product quality and user satisfaction can be improved. On the other hand, user experience is subjective, and different roles may have different conclusions on the same experience issue from different perspectives; at the same time, user experience is also dynamic, and the same experience can be evaluated in different environments and at different stages. The views on the problem will also be different; thirdly, experience issues need to maintain a certain dynamic balance with business development. Sometimes temporary compromises and sacrifices may have to be made, but compromises and sacrifices must not break through the bottom line of experience. Therefore, user experience goes hand in hand with business development, and as a quality assurance role, it plays a vital role in the process of user experience governance.

7.1.4 Quality process governance

Collaborate with PMO and production research to strengthen built-in quality and improve process quality, mainly including:

• Completeness of requirements

Through process specifications and regular reviews, we guide products to improve the quality of requirements and avoid one-sentence requirements, verbal requirements, and hasty reviews of requirements without sufficient consideration.

• Delivery specifications

Review and launch according to the rhythm, improve the rhythm of delivery, and minimize the interruption of testing and R&D; the R&D side avoids privately accepting business or product needs without reviewing or notifying the test in advance, etc.

• Proposal quality

The code quality of front-line development varies greatly. It is necessary to promote R&D to improve code quality through quality measurement analysis and improve the standardization of the development process, such as early code review, strengthening self-testing and single testing, etc.

• Production research test review

Regularly review problems in the demand delivery process and agree on the direction and progress of improvement.

7.2 Quality measures

Through the establishment of a quality measurement system, specific quality indicators and goals are formulated, and the results and improvement directions are measured through indicators. Taking the platform R&D department as an example, we selected a total of 6 indicators in 4 categories as the core indicators of quality measurement.

• Average number of defects per requirement/number of defects per R&D month: Used to measure the density of defects reflected in each requirement and for each developer.
• Defect closure duration/defect closure rate: that is, the time and proportion of time spent by R&D and testing on defect repair and verification, used to measure the speed and efficiency of resolving defects.
• Deployment rollback rate: used to measure the project team's control over system reliability and stability.
• Number of online incidents: used to measure the number of online incidents/problems at each level in the production environment.

Through year-on-month analysis, trend analysis and comparative analysis with the industry on these indicators, issues such as test submission quality and online quality are fed back, which is helpful for each department to identify problems and make timely adjustments to ensure continued high-quality delivery.

7.3 Quality culture construction

The significance of quality culture construction is to create and cultivate quality-focused values ​​and codes of conduct within a team, enhance team members' sense of participation and responsibility in quality, and promote continuous quality improvement. In the process of building quality culture, the focus is on the participation of all employees. Internally, in order to check for deficiencies in system functions, improve user experience, and promote exploratory testing concepts and ways of thinking, we carried out a "defect cleaning" activity. Due to the large number of teams and business lines that signed up, we adopted the method of pairing up the business lines as a unit. Business talks and scoring were conducted within the pairing groups, and each group controlled four to six people. In the end, a total of 233 BUGs were reported, and the testing team that was best at "finding faults" was selected. Common quality issues reported during the process include:

• User experience standards are not unified, and different people have inconsistent perceptions and identifications of defects, leading to disputes over the identification of some defects.
• Not enough attention is paid to operational experience issues.
• There are many inconsistencies in functions and experiences between the test environment and the pre-release and online environments.

In response to the above problems and defects discovered, test students have been arranged to follow up and provide feedback on the progress of the solutions. In addition, an event titled “Looking for the Most Beautiful Test Master—The Most Beautiful TestCase Arena” was also held around test case design. Each team writes test cases around the more complex product functions of recent iterations and submits them to the review office. In addition to strict evaluation standards, each judge also takes into account the betting votes of the audience outside the venue. With the unveiling of the "most beautiful test master", each test team also improved its own abilities based on the experience of the competition schedule.

7.4 Insight analysis and continuous improvement

In the quality management process, we have improved the BUG level and incident reporting system. According to the severity of defect problems, they are mainly divided into 5 levels, namely P0, P1, P2, P3, and P4, among which P0 is the most serious and P4 is the lightest. For daily online problems, each business line is required to provide timely feedback and record them uniformly in the event system, and organize review meetings for P0, P1, and P2 problems and report them to the next level. Each business line also needs to conduct internal reviews to discover the causes of online problems and summarize improvement measures and promote their implementation.

7.5 Summary, precipitation and sharing

Test students must not only continuously improve their testing professional capabilities, but also continuously deepen their understanding of the business. They also need to understand certain development technologies. The growth of professional capabilities, business knowledge and development technology needs to be achieved through continuous learning, summarization and sharing. Our team regularly organizes collective learning and special topic sharing. Each sharing needs to have video data saved, and the shared content will be synchronously produced as an article, so that other colleagues can learn and review at any time.

In addition, because there are so many projects and needs, various test students often need to provide support. In addition to the normal AB positions, they also need to have an understanding of other projects. Therefore, everyone jointly sorted out and continued to maintain the user growth test white paper, sorted out test-related content according to project dimensions, including test plans, test cases, regression strategies, test data, lessons learned, etc., and reviewed the test strategies, test methods, and data of some projects. The structure was deeply excavated and several patents were deposited. In addition, we regularly hold test lecture activities. Based on their understanding of the system, business, users and technical architecture, test students connect the business and system in their own language, lecture to product and R&D students, and ask questions at the same time , production, research and testing work together to analyze and discuss solutions. Through testing sessions, testing students not only deepen their understanding of the business and system, but also provide a lot of input for the optimization and improvement of product solutions and technical architecture, which can be said to serve multiple purposes.

In summary, quality operations are of great significance to enhancing team quality awareness and implementing quality process specifications. In our work practice, we usually arrange for two JDSs to take on the responsibility of quality operations, and generally need to allocate 10% to 20% of their time and energy to do the above quality operation-related work.

Summarize

This article aims to summarize the quality assurance methodology for user growth. Through the quality methodology formula of "high quality = diversification of quality strategies + standardization of quality processes + standardization of quality activities + platformization of quality tools + normalization of quality operations", the department's quality awareness, thinking, and Align behaviors and goals and provide guidance to quality assurance students in the department. At the same time, it elaborates on the connotation of user growth quality assurance values ​​and methodology. First of all, quality strategy diversification emphasizes formulating corresponding quality strategies according to different business needs and product characteristics to ensure satisfaction. Secondly, quality process standardization aims to establish a unified quality management process to improve work efficiency and quality. Third, standardization of quality activities emphasizes comprehensive planning and management of quality assurance activities, and access and exit of R&D activities through seven quality gates. Fourth, quality tool platformization advocates the use of advanced quality management tools and platforms to improve the level and efficiency of quality management. Finally, the normalization of quality operations emphasizes integrating quality management into daily work to achieve full employee participation and continuous quality improvement.

In the future, the quality assurance development direction of the user growth quality team will mainly include the following points. First of all, with the continuous development of AIGC technology based on ChatGPT, we are exploring the application and implementation of AIGC in the entire quality assurance process, including but not limited to business knowledge popularization, test strategy design, test case design, test code generation, and testing Data structure, etc. Secondly, user-centered quality management is becoming the mainstream trend. We need to pay more attention to user experience and feedback, use user needs and expectations as an important reference for quality management, and continuously optimize products and services. Finally, agile quality management will be further promoted and implemented. By adopting the agile development model and agile testing model, we can respond more quickly to users' growing business needs and user feedback, and achieve rapid iteration and continuous improvement.

Author: Jingdong Technology Wang Xianke

Source: JD Cloud Developer Community Please indicate the source when reprinting

 

The author of the open source framework NanUI switched to selling steel, and the project was suspended. The first free list in the Apple App Store is the pornographic software TypeScript. It has just become popular, why do the big guys start to abandon it? TIOBE October list: Java has the biggest decline, C# is approaching Java Rust 1.73.0 Released A man was encouraged by his AI girlfriend to assassinate the Queen of England and was sentenced to nine years in prison Qt 6.6 officially released Reuters: RISC-V technology becomes the key to the Sino-US technology war New battlefield RISC-V: Not controlled by any single company or country, Lenovo plans to launch Android PC

{{o.name}}

{{m.name}}