Ali sister REVIEW: Software quality is not measured out, but why there are so many test engineers for quality work? Testing is a cost department, what is the value of the test is to create? R & D model is constantly changing, constantly positioning the test of how to define the future of the test will be what form? Today, Alibaba Senior Test Development Specialist proud wild summarizes some thoughts on the future shape of the test, we want to do is inspire students tested.
Foreword
From the social development, the division of labor in all areas is getting smaller. But from the point of view of the development of the technology sector, the role of testing and development is in continuous integration, what is the reason behind? Iteration is the Internet faster and faster integration to promote multi-role, or because the technology (in particular the quality of technology) advanced productive forces gradually replacing the backward productive forces?
Before answering these questions, let's take a look back "Test Engineer" in the development process as a function or an individual past:
- 10 years ago, most of the primary output of the test piece is relatively one-time, such as a project to write text-based test, the basic project abandoned after release.
- Advanced testing of a methodology that period, such as the ability to test case design methods, project process management is very convincing already a high order.
- There are some technical ability to test students involved in the writing automated scripts. Automation "software" Testing the Internet era and the beginning, is the real hard core capabilities.
But such a test pattern and efficiency is very low, apparently unable to support the wave of the Internet without the fast is not broken. After 2010, the head of the enterprise team tested a series of changes quickly from the above-mentioned primary ability to expand to CI / CD technology-driven system, and ultimately drive the process of testing technology products form a clearer test platform development context.
In this period of nearly a decade, due to the innovative testing tools, platforms, test team got a breakthrough in the development. But the tool as an adjunct to traditional test mode, still encounter bottlenecks break. For example, from a global perspective quality has also undergone some branches:
- One is constantly insist on the development path of the platform: the quality of projects is based on continuous spawned all kinds of performance platform, but also to solve the problem itself, to expand research and development of various aspects from the traditional field of quality. Some large companies have begun to precipitate a common platform for collaborative research and development (R & D pipeline).
- One is to break out from inside: for example, Google's SRE team to purely technical means to create a built-in self-consistent and quality system (the traditional theory of falsification as a foreign-built quality system). [1]
Both the direction of the target, there is a certain overlap, for example, some companies in charge of the test line, to distinguish online SRE responsible. But if the quality of this large target, the future success of the picture should be: "The combination of quality and efficiency" and "outer built in conjunction with self-consistent." Because the only way to create a truly complete technical quality ecology.
Real-time quality
Also based on some of the above thinking and practice, we put forward the concept of "real-time quality" at the end of 2017. "It is not a specific test technology products, but a future-oriented ways and means to solve quality problems."
Its main features are: test run with real-time feedback can.
Why should we go in this direction?
With changing technology innovation and delivery model for the test team, we need to be transferred as quickly as possible to the real-time quality direction from the delivery-type quality. The traditional delivery model quality, the test as a hurdle to the task of arming the way in developing test lift, when the project release. In this way there is too much interaction between different roles, can only play a single point of quality assurance. The real-time quality objectives are: the quality of the means to modules, components and even systematic approach embedded in business applications, the ability to protect the quality of real-time form. Collaboration between developers and testers future (or will not distinguish between the development and testing), not just the synergy between people, more of the "code of business characteristics of the service," the two sides are complete and for the completion of "quality of service service Code "and cooperate with each other and form a system-level dependencies. On the quality of these systems provide, we hope various roles within the company can become a quality operator. Only do this, we may be testing the real from process-oriented to object-oriented.
Icon: ideal test work
Real-time quality of architecture
要做到质量的实时反馈和面向对象测试，这意味着我们的测试方法和协同方式发生了较为根本性的变化。我们需要以一个合适的方式参与到业务应用中，与此同时我们还需要把测试的各种能力封装成一个个服务，而不是现在的工具。工具终究是需要人来操作的，而我们希望未来测试任务的主体是机器、算法。测试人员只构建测试服务，而不参与测试过程，这也是最符合测试开发 Test Development Engineer 的 job design 。
图示：实时质量架构
那测试到底还需不需要做功能测试？可能在很长一段时间内仍然是需要的，但那一定只是日常工作中很小一部分。
实时质量是基于现有测试能力改造
我们在推进一个新的方向时，尽量不要去推翻重来。如果要面向未来，实时质量必须是可以向下兼容的，因为只是这样才能继承现有的测试沉淀，也才能被团队中的测试人员所接受和支持。只有自己不断进化才符合自然规律。所以我们需要更多强调对现有测试能力的改造，而避免另起炉灶。以下用运营页面测试的实时质量改造作为一个案例。
案例：运营页面的实时质量改造
作为电商域的同学对于运营页面应该非常熟悉，在之前也非常痛恨。比如：
“CBU的一次大促，运营人员至少需要配置千级以上的活动页面，而每一个页面上又包含几百上千个商品等活动元素，平均一个页面需要5到10分钟的人肉检测，同时运营和测试人员需要不断就测试标准和 Bug 来回讨论、提交。一次大促下来，我们至少需要十几人/日的测试资源才能保证会场的正确性。”
这个过程很痛苦，运营人员需要不断去找对应的测试同学协同，幸福感很差。而测试人员来说，这些页面的测试更多是一个重复劳动，一个黑盒。能力也得不到什么成长。我们如何对它来进行实时质量的改造呢？
总共分两步：
- 我们对传统的测试体系进行了改造。把以往通过人工测试的各个测试点，通过自动化的方式来实现。比如基于 DOM 树制定一系列规则，例如403这些的错误都可以被很好地扫描出来。同时，针对于一些无法通过规则排查的问题，我们运用了算法能力。例如空坑检测，一致性检测等。
- 把以上测试组件，通过消息的方式跟运营页面发布系统对接。
它的系统依赖关系是如下的：
图示：运营页面检测系统依赖图【示意】
同时针对于不同的业务场景，我们开发了不同的页面检测能力，比如针对于 DOM 树的页面检查：
还有基于算法能力的识别能力：
通过上述的改造后，对于运营人员发布页面以及页面的测试就极简化为三步一站式的能力。从以往运营、测试、开发之间的来回交接，变成了运营跟系统之间的交互。不仅提升了运营人员的页面搭建体验，也极大地提升了测试的效率。
在某次运行中活动中实际的执行结果【示意图】：
以上的过程和结果数据，也充分体现了“运行含测试，实时可反馈”的价值。
数据和算法是实时质量的核心
测试出现以来，我们一直习惯于代码逻辑类的测试，但数据一直都是测试很重要的生产材料。因为人肉执行任务的局限性，我们发明了等价类和边界值等测试理论和方法来用尽可能少的成本来尽可能多的验证问题。但一方面算法的不断应用，每一个数据都可能存在个性化的业务表达，我们可能无法找到一个通用的预期结果较验（还是会有一些通用的预期结果的，比如非空判断和区间等，但这类的预期不能很好地做业务判断）。因此，我们也需要用数据和算法能力来武装自己。
在以数据驱动的业务发展进程中，我们的测试主体已经从简单的代码转变为数据+算法。或者说，业务对质量的核心述求，已经从简单的页面错误、代码 BUG 到数据的准确性、算法的有效性（我老板在每次大促前，都要再三叮嘱我数据不能错）。如何来感知质量风险，以及捕获各类的异常？那必须先把数据、流量、监控来做收口，同时提升测试工具在大数据分析上的能力。
基于这些思考，我们构建了全域实时数据校验能力，是一款通过实时获取线上 DB 中的海量业务数据，完成业务数据校验、质量风险感知的产品。
案例：Captain 全域实时数据校验
图示：数据对比框架【示意】
它具备的一些能力：
- 严格的安全策略。
- 实时获取线上数据：通过强大的数据支持能力，平台可以在无损线上数据库表的前提下，通过 SQL 查询获取线上 DB 中的真实业务数据，且做到了实时获取，通过数据可以进行完善健壮的数据校验，从根本上提高对于业务的把控。
- 多样的数据获取方式：目前平台支持多种数据获取方式：单库单表查询、单库多表联表查询、分库分表查询、跨库的多表的联表查询。
- 多种比对方式支持，比如跨库查询和联表查询等等。
最主要，它可以用一套脚本无损地支持测试环境、灰度、生产环境等。让线下测试的所有经验可以得到复用和沉淀。（我们内部调侃说，这才是带着测试的灵魂的，而其他的很多产品都只是一个面向开发的工具）
在前期解决数据一致性，对账等常用的基本需求上，我们可以依赖于这些数据和测试的服务，展开更多的业务形态。
实时质量需要不断突破测试的边界
测试的边界在哪里？
过去有人告诉我，不能去修改业务应用的代码，只能让在盒子外面或者调用的方式来测试。还有人说，我们只开发工具，不能接触任何的业务。现在这些都在逐渐模糊，大家努力一起，让测试的很多活动，从简单的功能测试，往研发工具和业务质量等或前或后地迁移。
在过去的一两年，我们团队也已经慢慢承接了更多的职责，有些甚至于是直接服务于客服、运营和产品人员的。我认为，一支强的团队一定是不断走在突破原来工作边界的道路上。没有什么是一成不变的。
但每个职能团队都是有自己的核心价值的，而至于哪些应该由测试来做，哪些由开发做。我们的标准是：判断这件事情是更为了“让技术更有品质”还是“让技术创造新商业”？（“让技术更有品质”是我们团队的使命，“让技术拓展业务边界”是开发团队的目标）
以下虽然是几年前的例子，但也很好的体现了我们在边界的突破，以及如何用实时质量的思想来开装自己，创造提交 BUG 以外更多的价值。
案例：Offer 360提升客服端实时质量能力
Product link complex, large online troubleshooting difficulty, before developing into an average of 2-3 hours to deal with the problem every day online, but in fact most of the problems are normal business logic, and allows fully booked or technical support for self-inquiry of. Therefore, we provide real-time access by service error log and debug information, the user feedback to troubleshoot the problem, open to customer service. Help them the first time to solve customer problems.
Real-time quality of future planning
Real-time quality is an idea, I think it is coming across on the current development of two different branches.
Testing has been weakened over the years, I have seen a lot of good group leader transformation test development, product. If we do not think more and more, more and more exploration. If the tests do not dream, then what's the difference with salted fish?
Illustration: the future development of test
postscript
Last week, a development expert to see the message on the inside of a forum, or winter ridge feeling of. We have been emphasizing testing capabilities continue to evolve, emphasizing development capabilities, but the test of early heart can not be lost. We continue to improve on the tools, testing capabilities, but from the perspective of human and organizational point of view, in the pursuit of the most efficient at the same time, we need some organization designed to form mutual supervision between jobs. This is also a test phase 1.0 started testing a responsibility to be given.
Original release time: 2019-06-11
author: Proud wild
article from Yunqi community partners, " Ali technology " for information may concern " Ali technology ."