探索性测试与脚本测试:谁赢了?

目录

Context-based (Exploratory Testing) vs Scripted Testing Teams

What does it mean?(这意味着什么?)

Conclusion(总结)


Real-world benefits of exploratory testing(探索性测试的益处):

Traditionally, software testing has been a very rigid activity, but in recent years there’s been a shift away from script-based testing. Exploratory testing, which is more context-driven, has come to the fore. That’s because it gives testers more freedom to exploit their skills and knowledge, and it makes them responsible for optimizing the value of their own work. 

传统上,软件测试是一项非常严格的活动,但近年来已经从基于脚本的测试开始转移开了。 探索性测试,更偏向上下文驱动,已经脱颖而出了。 这是因为它让测试人员更自由地利用他们的技能和知识,并使他们有责任优化自己的工作价值。

                 

Not everyone is sold on the value of exploratory testing. The perceived lack of formality and emphasis on personal responsibility can set alarm bells ringing. But that concern is largely based on a misinterpretation of exploratory testing. It’s not about throwing rules out the window and testing at random, it’s actually very structured and systematic. And it’s also highly effective. 

不是每个人都认可探索性测试的价值。 人们认为缺乏形式和强调个人责任可能敲响警钟。 但这种担忧很大程度上是基于对探索性测试的错误理解。 探索性测试不是抛弃规则,并随机测试,它实际上是非常有条理和系统化的, 而且它也是非常有效的。

Skeptics want concrete proof that it does more than improving tester’s morale. That’s why we decided to conduct a study that would pit context-based, exploratory testing directly against a script-based testing approach. The results were very interesting, as you’re about to find out.

怀疑论者想要具体的证据:探索性测试不仅仅提高了测试人员的士气。 这就是为什么我们决定进行一项研究,将进行基于上下文的探索性测试,与基于脚本的测试进行比较。 正如你即将看到的,结果非常有趣。

Context-based (Exploratory Testing) vs Scripted Testing Teams

基于上下文(探索性测试)与脚本测试团队

Two teams, two approaches(两个团队,两种方法):

We started by dividing the testers into two teams of three. Testers in each team had the same comparable application knowledge. The same definitions for defect severity (major, minor) were established for both teams. Both teams had the same application build delivered to them. One team (“scripted”) would apply a traditional script-based testing approach and the other team (“exploratory”) would adopt a context-driven testing approach. The testing activities would be divided into two phases of three days each.

我们首先将测试人员分成两组,每组三人。 每个团队的测试人员具有相同的应用知识。 对两个团队确定了相同的缺陷严重性(主要,次要)定义。 两个团队都向他们提供了相同的应用程序。 一个团队(”脚本化”)将应用传统的基于脚本的测试方法,而另一个团队(“探索性”)将采用上下文驱动的测试方法。 测试活动将分为两个阶段,每个阶段为期三天。

The script-based team identified five business work-flows to test and generated 15 test cases. The test cases were limited in scope, so testers didn’t have any freedom to explore outside the confines of the script.

基于脚本的团队确定了五个业务工作流程来测试和生成15个测试用例。 这些测试用例的范围是有限的,因此测试人员无法在脚本范围之外进行探索。

The exploratory team created two visual mind maps, one that identified the test coverage and test charters, and the other covering product components/modules. The process produced 24 test charters in total. The charters defined were high-level and allowed contextual interpretation, extending the scope of the test session for the testers.

探索团队创建了两个视觉思维导图,一个识别测试覆盖范围和测试章程,另一个覆盖产品组件/模块。 该过程总共产生了24个测试章程。 定义的章程是高级别的,并且允许进行上下文解释,这样扩展了测试人员的测试范围。

Phase 1(阶段1):

The scripted team managed to complete 6 test cases in the three days allotted. They reported 6 major defects in that time.

脚本小组在分配的三天内成功完成了6个测试用例。 他们报告了当时发现的6个主要缺陷。

The exploratory team managed to complete 13 test sessions ranging from 30 minutes to 180 minutes each. They reported 10 major defects and 5 minor defects.

探索团队设法完成了13次测试,每次测试时间从30分钟到180分钟不等。 他们报告了10个主要缺陷和5个次要缺陷。

                           

Interestingly the exploratory team reported all of the defects that the scripted team had reported.

有趣的是,探索团队报告了脚本团队报告的所有缺陷。

Phase 2(阶段2):

The scripted team managed to complete 9 test cases this time. They reported 10 major defects and 8 minor defects.

脚本团队这次设法完成了9个测试用例。 他们报告了10个主要缺陷和8个次要缺陷。

The exploratory team completed 18 sessions. They reported 14 major defects and 5 minor defects.

探索小组完成了18次测试。 他们报告了14个主要缺陷和5个次要缺陷。

                                   

In phase 2, the scripted team reported 2 major and 1 minor defect that the exploratory team didn’t find, but the exploratory team reported 3 major and 1 minor defect that the scripted team didn’t report.

在第2阶段,脚本团队报告了探测团队没有找到的2个主要缺陷和1个小缺陷,但是探索团队报告了脚本团队没有报告的3个主要缺陷和1个小缺陷。

This doesn’t take into account the relative complexities of the workflows that may have been chosen by the testers within these sessions and the test cases, but we can still draw some interesting conclusions.

这并没有考虑到测试人员在这些测试用例中,可能选择的工作过程的相对复杂性,但我们仍然可以得出一些有趣的结论。

What does it mean?(这意味着什么?)

It would appear that an exploratory approach, and the responsibility and flexibility it engenders, results in a more effective form of testing. It may be possible to cover more ground by developing and adapting your test charters as the test sessions progress, based on what makes sense in context. This freedom is lacking in script-based testing and it can prevent defect discovery.

看来,探索性方法,及其产生的责任和灵活性,带来了更有效的测试形式。 通过在测试会话进展的基础上开发和调整测试章程,并基于上下文中有意义的内容,可以覆盖更多的内容。 基于脚本的测试缺乏这种自由,因而会不利于缺陷的发现。

Sticking rigidly to scripts creates well-worn paths, and it’s only by deviating from those paths that we’re going to uncover all the defects. As mentioned several times by thought-leaders within the testing community, “If you imagine a product as a field of landmines and each landmine is a defect, then it’s pretty clear that treading the same path over and over is not the way to find them all.”

严格遵守脚本导致测试的路径固定,只有偏离那些固定的测试路径,我们才可能发现所有缺陷。 正如测试界的思想领袖多次提到的那样,“如果你把一款产品想象为一个地雷领域,而每个地雷都是一个缺陷,那么很明显,一遍又一遍地走同一条道路并不能全部找到它们。”

In the end, neither approach was perfect, because each team reported defects that the other team did not identify, even if the exploratory team did report more, overall.

最后,这两种方法都不是完美的,因为每个团队都报告了另一个团队没有发现的缺陷,即使探索团队在总体上确实报告了更多缺陷。

Realistically, this may mean that the right approach, with regard to coming as close as possible to “minimal” defects, is going to be a mixture of the two. But, there are many benefits with the context-driven approach that speak in its favour. It requires less preparation time, less documentation, identifies issues earlier, and challenges testers to use analytical skills and deductive reasoning. They gain a deeper and more thorough understanding of the product and really act as advocates for the end user.

实际上,这可能意味着,想尽可能遗留“最少”缺陷的正确方法将是两者的结合。 但是,上下文驱动的方法有很多好处,有利于将二者结合。 它需要更少的准备时间,更少的文档,更早发现问题,并促使测试人员使用分析技能和演绎推理能力。 他们对产品有了更深入,更透彻的理解,并真正成为最终用户的倡导者。

Conclusion(总结)

The end result demonstrates that exploratory testing does lead to reporting of more defects before go-live, resulting in a better product delivered by the team, and ultimately, more satisfied/fulfilled testers which are all desirable outcomes, any way you look at it.

最终结果表明,探索性测试确实会在上线之前报告更多缺陷,从而使团队交付更好的产品,并最终获得更满意/更有成就感的测试人员,这些是理想的结果。

About the Author(关于作者)

Mush Honda is QA Director at KMS Technology, a provider of IT services across the software development lifecycle with offices in Atlanta, GA and Ho Chi Minh City, Vietnam. He was previously a tester at Ernst & Young, Nexidia, Colibrium Partners and Connecture. KMS services include application management, testing, support, professional services and staff augmentation.

Mush Honda是KMS Technology的QA总监,KMS Technology是一家跨越软件开发生命周期的IT服务提供商,在佐治亚州亚特兰大和越南胡志明市设有办事处。 他以前是Ernst&Young,Nexidia,Colibrium Partners和Connecture的测试员。 KMS服务包括应用程序管理,测试,支持,专业服务和人员扩充。

原链接

https://www.softwaretestinghelp.com/exploratory-testing-vs-scripted-testing/

猜你喜欢

转载自blog.csdn.net/wodeyijia911/article/details/87827672