Detailed agile testing best practices from one instance

 

Agile software development is very popular, and gradually extend the software industry's development model. Unlike traditional software development model, agile development model has its own distinct values ​​and methods. Among them, the agile testing is also different with the previous part of the software testing process. This raises new requirements for testers, it has brought new challenges. This will combine an instance of a software project, based on the different stages of project development, detailing the activities of each stage of the main test. Wen will analyze the prerequisites and main objectives and tasks of each test activity, and recommend the best solution according to the example.

 

Part I: Introduction to Agile Software Development

Agile Software Development (Agile Software Development) beginning in the mid-1990s. The earliest is for comparison with the traditional waterfall software development model (waterfall model), so when the method is called lightweight methods (Lightweight methods). The early twentieth century, 17 advocates the establishment of a method of the Agile Alliance (Agile Alliance), and name the software development methodology to agile software development process.

Agile Alliance summed up in the early days of the value of four basic principles:

1. Personnel exchanges over processes and tools (Individuals and interactionsover processes and tools)

2. Software products is more important than long-winded (Working softwareover comprehensive documentation)

3. Customer collaboration is more important than contract negotiation (Customer collaborationover contract negotiation)

4. Adaptable to behave weight (Responding to changeover following a plan)

Based on these four-point principles, agile software development has its own unique process (see Figure 1).

Figure 1. Agile Software Development Process

The whole process is interspersed many software development methods already appears before agile development, extreme programming (Extreme Programming, 1996), Scrum (1986), characterized in driver development (Feature Driven Development), test-driven development (Test Driven Development) Wait. These methods are fully reflected and applied at all stages of agile software development process.

For example, Scrum focuses mainly on project management, project manager (Scrum master) team need to develop customer needs in each cycle comes Sprint, the definition of the objectives of each Sprint, assign tasks, supervise and, finally, gains and losses and start plans new Sprint.

Instead, the driving characteristics developed and tested primarily used Sprint driven development cycle. If the project were to develop new features during this phase of the implementation of the main features of driver development. All testing and development personnel will focus their work on the new features above, both from development and testing to complete their task. If the project were to test new features during this phase of work will need to focus moved up test. All the testing and development staff are closely watching the current version of the defective condition. Testers need new defect cases on a daily stand-up meeting (Daily Standup Meeting) on ​​a business day before the report found, the project manager to decide whether to fix these problems progress of the project and defect severity. The need for timely repair of the defect is Sprint a new task in, the project manager will add to the Sprint Backlog and inform developers to fix bugs.

For agile development and testing of the review process, peer review (peer review) Extreme Programming ideas have been fully applied. In pursuit of code review and documentation is simple and efficient. Team members two each on a pair of mutual evaluation; sometimes, a developer and a tester can also be composed of a pair collaborate with each other. Such problems can contribute to defects at a first time and be denied in the bud.

Agile development there are several key concepts (Key Issues):

1. Iteration (Iterative process)

2. User stories (User stories)

3. Tasks (Tasks)

4. Standing Conference (Stand-up meeting)

5. Continuous integration (Continuous integration)

6. The Minimalist Program (Simplest solutions)

7. Reconstruction (Re-factoring)

These concepts are often used to agile development ideas and methods. Below we will discuss in detail testers play in agile software development roles and functions.

 

Part II: testers in agile development

This section briefly describes the agile development testers need to have the qualities and responsibilities.

2.1 Agile Team

Our agile development team of four developers, two testers, a product design, a project manager and a product manager composition (see Figure 2). Ten in the morning every day, at a fixed time and meeting rooms inside, the team will hold meetings standing. At this time, team members report the day before each task to the project manager in accordance with the established order, and the difficulties encountered in the day to complete the task. At the same time, the project manager to update Sprint Backlog (a well-made Excel spreadsheet), and solve everyone's problems raised.

Figure 2. Agile development team members

Because agile development requires participants to quickly and efficiently respond to changes too, so virtually make high demands on the tester.

2.2 testers need to have the quality

Testing is an integral part of software development. It is also true in agile software development. Different organizations to testers with different names: such as test development (Test Developer), Quality Analyst (Quality Analyst), Software Quality Engineer (Software Quality Engineer).

Each title implied different functions. Above corresponding title capacity following requirements:

1. Quality control and ability to write code -> test development

2. Preventing defects (Quality Assurance) and QC (Quality Control) capability -> Quality Analyst

3. Ability to develop and execute test programs -> Software Quality Engineer

In conclusion, there is the basic quality requirements in three areas: coding (Coding), testing (Testing) and analysis (Analysis).

In many other development process, each stage has been tested the ability of different testers; sometimes focus analysis (such as system configuration test), sometimes focusing on coding (such as functional testing). However, in the agile development process, testers need to combine these three areas to work, the only way to truly reflect the essence of Agile Testing: Simple and efficient must respond to change.

The main duties of 2.3 testers

In agile software development, the role of testers has three main aspects:

1. Definition of quality (Define Quality): This should be the basic responsibility of software testers. Agile methods encourage testers when Sprint plans to communicate directly with customers, from their own experience, to develop common quality requirements for the product features.

2. AC defect (Communication): Agile processes emphasize the exchange team. Developers often focus on the important and novel feature, testers should seize the details, look for the design of "missing door"; In addition, developers using unit tests to ensure the basic quality of the product, the tester can use the acceptance test (Acceptance Test) to identify inconsistencies between customer needs and the actual results.

3. Timely feedback (Feedback): stressed agile process simple and efficient. Testers need for timely feedback current product quality issues. As a result, the team can only be addressed immediately. If the traditional process is a summary of the state, then once a week, agile process requires a summary of quality problems every day. In our project, the internal test report will appear on the internal site in the form of a web page. Each team member can get at any time. In addition, our testing framework provides self-test (Self-assistant Test): by clicking on the list of test cases in a specific use cases, developers do not need to interrupt the work of the tester can reproduce the defect.

Above summarizes the ability of testers in agile development needs and show the tasks, please follow the following example of a project to learn more about best practices for agile testing.

 

Part III: Agile Development in the testing process

This section combined with a software project, details of the main test process activities, provided that each activity and target tasks and so on.

3.1 Introduction Project examples

Project Description: According to an online B2B company requirements, we will develop a similar Google for its search service. As a Web Service, the service can be embedded in web pages. When the user inputs a keyword and select the type and location of a business, the system returns a list of specific merchant (see FIG. 3).

Figure 3. Project example of FIG.

Typical Agile development and testing activities see table below. It mainly consists of three parts, from the initial design and publish user stories to the plan, several Sprint cycle iterative development and testing, as well as the final product release phase. Each time period has a corresponding test activities. Sprint period is usually divided into two categories: the characteristic period (the Feature Sprint) and release cycle (Release Sprint). Characteristic period mainly related to the development and testing of various types of new features. Release cycle will be combined with programs to determine the function of the new version, and then test the latest features.

The main activities of agile development	Test event
User Stories Design	Looking for hidden assumptions
Release plan	Summary of the acceptance test case design
Sprint iteration	Acceptance testing time estimate
Coding and unit test	Estimate the testing framework built
Reconstruction	Detailed design acceptance test cases
integrated	Write acceptance test
Acceptance test	Reconstruction of acceptance testing
Sprint ends	Acceptance test
Next Sprint start	Regression testing
release	release

In Sprint iteration cycle, the developer and the coding unit may be divided into portions according to conventional testing step, remodeling and integration. It should be noted that the reconstruction and integration is the Sprint iterative agile development tasks can not be ignored. If you want to be optimized to the last function in the new Sprint cycle and improvement, reconstruction and integration can not be separated.

Sprint before the end of each cycle, the test team will be submitted for acceptance testing period or the last Sprint Sprint cycle function has been completed (in actual projects, the progress of the testing team usually later than the development team). As a result, development teams can run the acceptance tests to verify whether the feature is currently being developed in line with expectations. Of course, this is expected to continue to change and improve in iteration.

When all the features of the product can be achieved, the test work has basically ended, entered the release cycle. At this point, the test team 's task relatively high.

Above, we outlined the main activities of agile development. Here we will make a detailed description and analysis of the various stages of the corresponding test activities. First, users design and publish the story stage.

3.2 Design and publish user story planning stage

在用户故事和发布计划阶段，项目经理和产品经理会根据客户的需求，制定概要的产品发布日程计划。此时，测试人员可以和开发人员一起学习新的功能，了解客户的需求。其中，有两个主要活动：寻找隐藏的假设和设计概要的验收测试用例。

3.2.1 寻找隐藏的假设

正如前文所述，开发人员通常关注一些重要的系统功能而忽视细节。此外，敏捷开发倡导简单的实现方案，每个开发 Sprint 周期不可能将功能完美得实现；相反，每个 Sprint 都会增量得开发一些功能。所以，测试人员在最初就需要从各种角度来寻找系统需求，探索隐藏的假设。

项目实例：

1. 从在线 B2B 公司角度思考

Q：这个搜索框对公司的业务有什么价值？

A：搜索框可以为用户方便得提供商户的目录信息。如果越来越多用户使用这个搜索框，可以增加我们网站的访问量。

1. 从用户角度思考

Q：作为查询信息、寻找商业合作伙伴的网站用户，搜索框对我有什么好处？

A：坏处：找到一家商户的地址，过去才发现已经关门歇业

好处：查找商户很简单，只要轻点鼠标

不快：有时候在寻找一类商户，却记不清楚具体名字

1. 从程序员角度思考

Q：一个搜索框的最简单实现方法是什么？

A：一个有 text input 和 search button 组成的 form；后台通过 server 程序将符合类型和地址的商户名从数据库中取出，返回给用户；每个返回项包括商户的名称、地址和评价意见。

1. 寻找这些观点中的问题

Q：搜索框如何在用户忘记具体名字的时候提醒用户？

A：在第一版本中实现比较困难。可以让用户输入至少一个类型来提高模糊查找的效果。

1. 最后寻找到隐藏的假设

以上的思考让测试人员对系统的隐含假设更加清晰：

首先，系统应该能够在高峰时候处理 200 条搜索请求和 1000 个鼠标点击事件。

其次，用户可以在已经查找到的内容中继续查找

最后，系统提供一个商户类别清单；如果用户选择商户类别而忘记具体名字，系统提供模糊查询。

在敏捷开发中，这些假设可以作为用户故事记录下来，从而指导未来系统的开发和测试。

3.2.2 设计概要的验收测试用例

定义完一系列用户故事后，测试人员就可以着手设计概要的验收测试用例。正如我们在前文论述，不同于单元测试，验收测试检查系统是否满足客户的预期，也就是用户故事是否能够实现。于是，测试人员可以根据每条用户故事来扩展，寻找其中的“动作”，然后为每条“动作”制定正例和反例。

项目实例：

动作	数据	期待的结果
搜索	一组能成功搜索到的（类别，位置）数据	在该类别和位置条件下的一组商户信息
搜索	一组不能成功搜索到的（类别，位置）数据	空列表

3.3 迭代 Sprint 阶段

当一个 Sprint 周期正式开始时，项目经理将制定该周期的具体开发和测试任务。在定期的 Sprint 计划会议（Planning Meeting）上，每位团队成员都要提供自己在未来一个 Sprint 周期中的休假和培训计划。另外，每个团队可以根据各自团队成员的能力和工作经验，适当设定一个工作负载值（Load Factor）。比如，我们团队的工作负载值为 75%，也就是说每个人平均每天工作 6 小时（以 8 小时计算）。接着，大家就可以开始分配任务。

当开发团队开始编码和单元测试时，测试人员的工作重点包括：估算验收测试的时间、估算测试框架的搭建、详细设计验收测试和编写验收测试代码。第两个主要活动一般在项目初期的 Sprint 周期中完成。其他的三个主要活动将在接下来的多个 Sprint 周期中视情况迭代进行。下面我们将具体介绍每个主要活动。

3.3.1 估算验收测试时间

在软件开发初期，需要估算时间以制定计划。这一点在敏捷开发中应用更加广泛。如果以前的开发模式需要测试人员估算一个软件版本发行的计划（这样的计划通常会延续几个月），那么现在则要在每个 Sprint 机会会议上估算两周到一个月的任务。此外，在每天的站立会议上，测试人员需要不断得更新自己的估算时间，以应对变化的需求。所以，每个测试人员都应该具备一定的估算任务能力。下面我们将介绍两个通用的估算测试计划的方法：

1. 快速而粗糙的方法

从经验而言，测试通常占项目开发的三分之一时间。如果一个项目开发估计要 30 天 1 人，那么测试时间为 10 天 1 人。

项目实例：

搜索框的开发估计需要 78 天 1 人完成。但是，考虑到系统有模糊搜索的功能，所以测试任务可能会占 40％左右，大概 31 天 1 人。下面列出了具体的任务：

任务	估计时间
设计测试用例，准备测试数据（搜索数据集）	8
加载数据集	2
编写自动测试代码	18
执行测试和汇报结果	3
总结	31

1. 细致而周全的方法

这个方法从测试任务的基本步骤出发，进行详细分类。其中包括 :

1. 测试的准备（设计测试用例、准备测试数据、编写自动测试代码并完善代码）

2. 测试的运行（建立环境、执行测试、分析和汇报结果）

3. 特殊的考虑

项目实例：

估算单个测试任务的事例参见下表：

测试	准备	运行	特殊考虑	估算
1	设计测试用例	0.5	建立环境	0.1
	准备测试数据	0.5	执行测试	0.1
	编写自动测试代码	0.5	分析结果	0.1
	完善自动测试代码	2.5	汇报结果	0.1
总共		4		0.4	0	4.4

估算多个测试任务的汇总参见下表：

测试任务编号	准备	运行	特殊考虑	估算
1	4	0.4	0	4.4
2	4	0.4	0	4.4
3	12	4.5	8.5	25
4	4	0.4	0	4.4
5	4	0.4	0	4.4
6	4	0.4	0	4.4
7	4	0.4	0	4.4
总共		51.4

3.3.2 估算测试框架的搭建

测试框架是自动测试必不可少的一部分工作。由于敏捷开发流程倡导快速而高效得完成任务，这就要求一定的自动测试率。一个完善的测试框架可以大大提高测试效率，及时反馈产品的质量。

在敏捷开发流程中，在第一个 Sprint 周期里，需要增加一项建立测试框架的任务。在随后的迭代过程中，只有当测试框架需要大幅度调整时，测试团队才需要考虑将其单独作为任务，否则可以不用作为主要任务罗列出来。

项目实例：

考虑该项目刚刚进入测试，需要为此建立一个测试框架。于是，在原先的估算中多增加一些任务。

任务	估算（小时）
选择测试工具	3
建立测试系统	3
编写下载、存放和恢复测试数据的脚本	2
寻找或建立测试结果汇报工具	8
设计具体的搜索测试用例	4
准备搜索测试数据	4
编写和测试“搜索”模块	3
编写和测试“验证返回列表”的模块	1
学习“在结果中搜索”的模块设计	4
编写和测试“在结果中搜索”模块	4
第一次执行测试	4
分析第一轮测试结果	4
第二次执行测试	4
分析第二轮测试结果	4
总共	52

3.3.3 详细设计验收测试用例

完成对测试任务的估算，接着就可以着手详细设计验收测试用例。我们可以对概要设计中的测试用例进行细化，根据不同的测试环境、测试数据以及测试结果，编写更详细的测试用例。另外，可以结合几个用例，完成一个复杂的测试操作。

由于敏捷开发的流程是不断迭代的过程，所以很多复杂的功能可能会在未来的 Sprint 周期中被优化。对测试人员而言，一个有效的方法是尽量将一些验证基本功能的测试用例作为基本验证测试用例（Basic Verification Test Case）在第一时间实现自动化；而对一些复杂的功能测试用例，可以先采用手工的方法测试，直到在未来 Sprint 周期中该功能达到稳定时候再考虑自动化。此外，对测试中出现的缺陷可以设计回归测试用例（Regression Test Case），为其编写自动测试代码，使得此类问题在发布周期（Release Sprint）时可以顺利而高效得进行验证。

项目实例：

基本验证测试用例：

动作	数据	期待的结果
登录	用户名：（空） 密码：（空）	“用户名和密码无效”

功能测试用例：

动作	数据	期待的结果
登录	正确的用户名和密码	进入系统：请输入搜索条件并点击“搜索”按钮
搜索	错误的类型	提示正确的类型
搜索	使用正确的类型	商户列表

3.3.4 编写验收测试用例

敏捷开发不提倡撰写太多的文档，提倡直接编写测试用例。此外，测试人员和客户应取得良好的沟通，将这些需求总结下来，转化成验收测试用例。如果资源充足，最好对验收测试用例建立版本控制机制。

考虑到需求在每一轮 Sprint 周期中会不断得变化，测试团队要控制测试的自动化率，正确估计未来功能的增减。自动化率过高会导致后期大量测试代码需要重构，反而增加很多工作量。

3.4 Sprint 结束和下一个 Sprint 开始

在一个 Sprint 周期结束时，团队要举行一个回顾会议（Retrospective Meeting）。团队成员可以在会议上畅所欲言，指出在过去一个 Sprint 周期中可行的，不可行的和有待改进的地方。待改进之处将在项目经理监督下于未来的 Sprint 周期中实现。

由于敏捷开发倡导增量开发，当新的 Sprint 开始时，测试团队需要根据新 Sprint 周期的开发进度及时重构验收测试。如果新 Sprint 周期没有具体的新功能开发，测试团队可以将精力集中在执行验收测试和寻找缺陷上。

如果下一个 Sprint 周期是发布周期，那么测试人员需要准备执行回归测试。下面我们来详细了解每个测试活动。

3.4.1 重构验收测试

正如上文所提及，敏捷开发是以迭代方式进行的，功能在每次迭代中推陈出新。于是，验收测试用例经常需要修改或者添加，相应的验收测试代码也需要删减。这部分工作如果时间花销很大，最好在估算的时候一并提出。

项目实例：

在下一个 Sprint 周期中，我们需要实现之前没有实现的“模糊查找”功能。测试人员要在新的 Sprint 周期中更新原来的验收测试用例，在测试“搜索”模块中添加模糊查找测试。重新估算的测试任务参加下表：

任务	估计时间
设计测试用例，准备测试数据（模糊搜索数据集）	2
加载数据集	1
编写自动测试代码	3
执行测试和汇报结果	2
总结	8

3.4.2 执行验收测试

验收测试可以分为两大类，基本验证测试和功能测试。如果是基本验证测试，推荐开发人员在运行完单元测试和提交代码前直接运行自动测试脚本。如果是功能测试，可以在每个 Sprint 后期，新功能代码提交后，由测试人员单独执行。

敏捷开发的开发和测试是相辅相成的。一旦基本验证测试出现问题，那就说明开发人员的实现违反了最初客户定义的需求，所以不能够提交。如果功能测试出现问题，那么测试人员要及时与开发人员沟通。如果是缺陷，需及时上报给项目经理，并在每天站立会议中提出；如果不是，那么继续下一项任务。这个过程充分体现了敏捷开发所提倡的团队交流机制。

3.4.3 执行回归测试

在发布周期中，测试人员所肩负的任务非常重要，因为这是产品发布前的最后质量检验。

首先，要建立一套自动生成 build、运行自动测试代码、手工执行测试用例并汇总测试结果的框架。估算方法参加上文。

其次，定期执行各类测试，包括功能和系统测试。

最后，要整理之前在每个特征测试周期中出现的问题。如果已经整理并归类为回归测试用例，那么只要定时执行就可以了；否则，需一一添加。如果用例已经被自动化，可以直接运行；如果是手工测试，测试人员需要按照测试用例进行操作，最后汇总测试结果。这部分测试就是所谓的回归测试。

 

总结

以上我们回顾了敏捷测试在整个项目开发中的基本流程。详细介绍了各阶段存在的主要测试活动，结合实际项目，叙述每个测试活动的最佳实践。

最后，我们来探讨一下测试中的两个问题：手工测试和测试报告。

手工测试和自动测试是两个主要的测试类型。考虑到敏捷开发的高效性，自动测试会优于手工测试。手工测试有两个主要的缺点：不可靠和容易被遗忘。比如，在文中的搜索实例中，一旦我们重新建立索引，那么先前在搜索文本中出现的文字错误就无法重现。另外，当测试人员按部就班得手工完成一个一个测试用例时，他们很容易遗忘一些特殊的测试用例，很多缺陷因此而被埋没。敏捷测试主张一些基本的验收测试可以被自动化；对一些涉及系统方面的测试，手工测试比较适合。

测试报告是反映一个测试团队工作的最好成果。为适应敏捷开发的节奏，测试报告可以以网页的形式发布在内部的 web 服务器上，在一些问题区域上标注鲜明的色彩，用来警示团队中的每个人。

综上所述，本文详细谈论了敏捷开发中测试的各项任务。希望本文有助于正在使用敏捷模式或者打算使用敏捷模式的团队更好得理解敏捷测试。