Construction and application of quality management system

The open source China community team made its first live broadcast, telling the story behind the open source China community in the name of sharing."

1. Background

Quality assurance is a systematic project based on software testing and follows the law of progressive development. By formulating implementation strategies according to local conditions, designing scenario plans, obtaining test results, and repeating the cycle. In the end, with the joint efforts of every Dewu test engineer, a set of quality assurance methodology adapted to Dewu technology was accumulated, which is the Dewu quality management system introduced in this article.

2. Constructing the “Four Modernizations” System

After three years of construction, the Dewu quality management system now has complete mechanisms, processes , methods and tools . The goal is to ultimately form a three-dimensional material quality management system through the realization of standardization, online, automation and intelligence. in:

"Mechanism" means: (R&D) quality assurance mechanism . Based on the "iteration quality review mechanism", every iteration is carried out and the online risks are controllable before it can be released.
"Process" means: testing & collaboration process . It is divided into testing process and collaboration process. The application of test processes is used to reflect R&D quality impartially ; supplemented by test quality calibration. The effective application of collaborative processes will help improve test quality. Therefore, collaborative quality is introduced to stabilize test quality and ensure final online stability.
"Method" means: implementation method . With the "Quality Month" activity in the business domain as the main carrier, we continue to optimize the quality of the R&D process, including basic indicators such as cross-domain review, CR merger, on-time test submission, smoke pass, daily defect clearance, defect introduction, defect escape, etc.
"Tools" means: quality engineering tools . Focusing on stability, quality, efficiency, security, experience, and compliance as the core, we will improve tool support capabilities. Each tool will eventually point to a core item, such as: RDC is used to standardize, online and partially automate the R&D process to improve collaboration efficiency, thereby optimizing delivery "efficiency"; the quality dashboard serves as the quality of the R&D process Insightful observability tools for process “quality.” The existing tools (R&D tool chain) are shown in the figure below:

The "iterative quality review mechanism" is a unified review mechanism for the entire quality platform. Relying on the quality dashboard as the main tool, combined with the quality assurance portfolio strategy of each business domain, we focus on drill-down analysis of the quality and performance of each iteration. Identify the risks that affect delivery within the current iteration and the corresponding avoidance measures to ensure that the release goes online when the risks are low or the risks are controllable.
"Built-in quality" is commonly known as "R&D quality" and is officially solidified after testing. "Test quality" reflects the comprehensiveness of test coverage through defect escape levels, and at the same time, it can calibrate R&D quality. "Collaborative quality" is used to stabilize test quality and ensure final online stability. Examples are as follows:
- Failure to test requirements on time brings risks to the test execution progress;
- Frequent process changes bring about timing deviations, rework losses, and even missed change coverage.
The "Quality Month" special activity is comprehensively promoted in all business areas of Dewu Technology. Through the "pre-event" application of the cross-domain use case review mechanism, the upstream and downstream collaborative technology implementation and verification coverage scenarios are connected; the "in-process" application of the "iterative quality review mechanism" , effectively assess process quality risks, and strictly control the quality of accurate requirements; apply the "online event review mechanism" "afterwards" to form a result verification to guide the next iteration in the development, testing, release, and production stages. Details optimized and improved. After achieving comprehensive results, the keyword "quality" advocated by the event has long been integrated into the consciousness of every Dewu technical person, and high-quality delivery standards are practiced daily.
The relationship between "mechanism, process, method, and tool" is defined as the mechanism sets the overall framework and goals of quality management, and the process defines the specific steps to achieve these goals, and the method is the specific implementation technology adopted in each step of the process. , while tools are the auxiliary equipment or software required to perform methods and processes. These four concepts are interdependent and mutually reinforcing, and together constitute the overall system of R&D quality management, ensuring that every aspect of products and services from design to delivery can meet predetermined quality standards.

3. Objectives and Framework of the Mechanism

Quality is the foundation of the quality assurance mechanism, and stability is the natural result of effective quality activities. Through reasonable fault definition and classification as the traction target, we must comply with the principle of never causing major problems (faults), quickly recovering from small problems (smoke points/events), and minimizing the impact of faults as much as possible (fault impact = fault impact area x fault repair time ) principle to be verified.

With the rapid development of Dewu's business scale, Dewu App is facing severe challenges in terms of availability, capital loss prevention and control, privacy compliance, and data security. Dewu Technology divides and conquers based on business characteristics. At the same time, it downgrades production faults layer by layer to achieve the effect of breaking them into parts, and has achieved the success of a continuous decrease in the total number of production faults. As shown below:

The framework of the quality assurance mechanism is described as follows: using the quality objectives of the clarification stage of the Quality Management White Paper V2.0 to concretely define the corresponding execution specifications and supporting requirements. Apply iterative quality review during the iterative process, identify risks before approval and make drill-down attribution diagnosis, formulate specific measures for risk downgrade or risk avoidance, and implement them before going online; finally, through iterative quality review, restore the typical process On-site, draw inferences from one example; at the same time, use online operation results to calibrate/correct the adaptability of the targets and supporting specifications. The whole process is shown in the figure below:

Then, by repeating the above cycle, a spiral upward effect is formed, rhythmically progressing to higher goals.

值得指出的是，经过上百个迭代的优化，研发质量显著提升超60%，技术债务大量减少70%；过程基础质量指标，如准时提测率、冒烟通过率稳定在99%以上。

4. Process and specifications

Dewu project management uses Sponsor to distinguish between business and technical demand types; at the same time, it is delivered into iterations or projects based on requirements complexity, scale and other considerations. For four combinations, namely business iterations, business projects, technology iterations and technology projects. The testing process and specifications are slightly different. The quality assurance process and specifications for business/technical iteration requirements and the quality assurance process and specifications for business/technical independent projects (including small projects) are respectively formulated. The following figure shows the general testing process and specifications:

Among them, a quality assurance plan for quality platform business/technical transformation needs is specially developed, defining key test actions, standards, and tool platforms, and guiding the test team to carry out their work in detail, as shown in the figure below:

In the process, based on case-driven optimization and improvement, the implementation plan V2.0 of the cross-domain use case review process and standard quality platform cross-domain use case review mechanism was precipitated to avoid the non-coverage of boundary issues caused by horizontal collaboration of multiple business domains.

In addition, for business/technical projects, we formulate a quality assurance plan for quality platform business/technical transformation projects, and refine the responsibilities and execution specifications of the test owner. For the scenario of business brushing, develop a business brushing test plan, clarify the process specifications, and reduce the change risks caused by high-frequency business brushing changes.

5. Methods and Tools

Test cases are the most basic testing assets, just like code is to APIs and APIs are to distributed systems. With the help of various tool platforms, it can be transformed or processed into various forms of "test products". The application of various "test products" relies on the research and development in the Dewu business field and the implementation practice of test students. After three years of continuous accumulation, a hundred schools of thought contend and develop vigorously. Here are some typical practices:

Interface automated testing, as the main method of regression testing, is also a classic method to improve test execution efficiency. Through the interface automation platform , it is applied to the development self-test, smoke (admission) and integration regression stages. ROI shows a linear relationship. For high-frequency iteration mode, automated testing can ensure the availability of core links; especially in service/interface migration, architecture governance and other technical transformation projects, revenue is maximized to protect technical debt settlement.
Traffic playback, as a test method before going online, is used in the pre-release stage through the traffic recording and playback platform . Through simple production recording and pre-release playback, a more realistic user scenario coverage can be obtained at a lower cost. Combined with interface coverage, select or structure incremental traffic to the traffic pool to make the distribution more comprehensive. Due to its low-cost application method, it can be used for bottom-up testing before release at low cost; in addition, it is extremely application-friendly for changes such as service/data migration and partial reconstruction, greatly reducing testing investment.
Precision testing, as an auxiliary test analysis and execution method, detects and fills gaps to improve test coverage and make measurements more accurate. Through the accurate testing platform , it is applied to the smoke stage (after testing), and based on code changes, it matches the maintained tags and binding relationships, such as asset loss fields, asset loss codes, (over-authorization) sensitivity, etc. Recommended test case execution coverage. If a discrepancy occurs, that is, there are no associated interfaces, CR is often required and the coverage plan is evaluated. Its application can not only verify the accuracy of change scope evaluation, but also evaluate the comprehensiveness of test analysis, forming positive feedback for the understanding of business/system modules.
Monitoring and alarming, as a method of offline auxiliary test execution, detects and fills gaps to intercept abnormal defects and measure in more detail. Through the monitoring platform , it is applied to the offline test execution phase. Development and testing students jointly formulate business alarm rules to achieve alarm aggregation and convergence; on this basis, alarm insights and defect transformation are realized, and be benignly applied in a T or T+1 daily clearing method.
Front-end inspection, as a proactive response method for user experience, daily inspection pays attention to user-side usage feedback to detect and repair early. Through the front-end inspection platform , the existing URLs are imported through the TOP access records of the past xx days, and scheduled task inspections are set; incremental imports are made through precise URL recommendations, and inspection coverage is provided. In the pre-release stage, JS Error exceptions can be intercepted in advance. In the production and operation stage, early detection and early intervention can be achieved, and proactive interception can be achieved before users.
Capital loss prevention and control Dcheck, as an effective method to intercept capital losses, provides real-time reconciliation and early detection and early hemostasis. Through the Dcheck platform , it is applied online and offline. Offline practice of asset loss scenarios can keep script rules fresh; online real-time monitoring and abnormal alarms.

In terms of tool application, it is worth noting that there are three costs that need to be continuously optimized to gradually reduce:

"Conversion" cost of test cases (development investment)
The cost of “supporting” each other between tools (integration effort)
“Use” cost of various tools (tool ease of use)

The figure below shows some efficiency optimization points and optimization ideas.

In the end, with the clear atomic capabilities of the tools and the convenient combination of applications, test engineers gradually focus on in-depth understanding of the business system and detailed review of technical implementation, thereby completing the formulation of reasonable test strategies and agile implementation.

6. Focus on “Keywords”

As the focus of the official plan, the "keywords" of Dewu technology are the basis for value judgment in carrying out complicated and complex work. They are also the central idea of stage reviews and summaries. They are also the outpost and beacon of effective business support, guiding technical work to be done correctly and properly. good. Begin with the end in mind, practice unremittingly around the "keywords", and continue to gain.

Stability is generally a natural consequence of effective quality assurance. It is often led by the stability or operation and maintenance team to conduct real-time online monitoring and emergency response to failures. The number of production failures and their distribution are the main approval indicators. The strategy follows: No major problems will occur, small problems can be recovered quickly, and the impact of faults should be minimized as much as possible, that is: fault impact = fault impact area x fault repair time.
Efficiency is supported and guaranteed from the bottom up through business delivery capabilities, planning support capabilities, and process collaboration capabilities to achieve efficient goals. That is, using data to present "more" and "fast" and combining NPS to fit the business experience.
Experience, oriented to users to improve the experience of using Dewu products. It makes the App easier to browse, the products easier to buy/sell, and the after-sales service more secure. For the Dewu App, we measure various aspects such as launch, initial launch, performance, package volume, security and privacy compliance, baseline deterioration prevention, and competitive product rating tracking.
Growth, software engineering is a discipline that emphasizes practice. Practice and continuous practice. The accumulation gained in practice is the gain of the growth of engineers. According to the Feynman learning method, think clearly, write it out, and share it through blogging.
Innovation, through "combination innovation" and "AI innovation", dig deep into stability and efficiency to find effective scenarios. In terms of stability, focus on comprehensive coverage and coverage granularity; in terms of efficiency, focus on implementation and operational reduction.

7. Summary

The mechanism sets the overall framework and goals of quality management, the process defines the specific steps to achieve these goals, the method is the specific implementation technology taken in each step of the process, and the tool is the auxiliary equipment required to execute the method and process or software. These four concepts are interdependent and mutually reinforcing, and together constitute the overall system of R&D quality management, ensuring that every aspect of products and services from design to delivery can meet predetermined quality standards.
The framework of the quality assurance mechanism is described as follows: Use the quality management white paper to clarify the quality objectives in the stage, and concretely define the corresponding implementation specifications and requirements; apply iterative quality review in the iterative process, identify risks before approval, and make drill-down attribution diagnosis , form specific measures for risk downgrade or risk avoidance, and implement them before going online; finally, through iterative quality review, restore typical sites of the process, and draw inferences from one instance; at the same time, use the online operation results to calibrate/correct the goals and the appropriateness of supporting specifications Matching nature.
Test cases are the most basic testing assets, just like code is to APIs and APIs are to distributed systems. With the help of various tool platforms, it can be transformed or processed into various forms of "test products".
With the clear atomic capabilities of the tools and the convenient combination of applications, test engineers can gradually focus on in-depth understanding of the business system and detailed review of technical implementation, thereby completing the formulation of reasonable test strategies and agile implementation.
As the focus of the official plan, the "keywords" of Dewu technology are the basis for value judgment in carrying out complicated and complex work. They are also the central idea of stage reviews and summaries. They are also the outpost and beacon of effective business support, guiding technical work to be done correctly and properly. good. Begin with the end in mind, practice unremittingly around the "keywords", and continue to gain.

* Text/ Bruce

This article is original to Dewu Technology. For more exciting articles, please see: Dewu Technology official website

Reprinting without the permission of Dewu Technology is strictly prohibited, otherwise legal liability will be pursued according to law!

Construction and application of quality management system

Guess you like