Comparative analysis of three interface testing tools: SoapUI, Jmeter and Postman

Reprinted: http://blog.csdn.net/huilan_same/article/details/75413482

Author: huilan_same

1.   How use cases are organized

The first is the difference in the organization of use cases. Different directory structures and organizations represent the testing ideas of different tools. To learn a testing tool, you should first understand its organization.

The organization of SoapUI is as shown below. The top layer is WorkSpace. Each window can only open one WorkSpace (this is an xml file), and each Project is also a separate xml file (for collaborative work, it can also be converted into A bunch of file collections), so each WorkSpace can open multiple Projects, and a Project can also be in different WorkSpaces.

Project corresponds to our test project, which can add WSDL, WADL resources, TestSuite and MockService.

TestSuite corresponds to our test modules, such as Merchant Center, where TestCase can be added, and TestCase corresponds to our different interfaces to a module, such as order management interface. An interface may require multiple steps to complete, and variables, data sources, requests, etc. are all one step.

The organization of Jmeter is relatively flat. First of all, it does not have the concept of WorkSpace. It is directly TestPlan, which is equivalent to the Project in SoapUI. The Threads Group created under TestPlan is equivalent to TestCase, and there is no TestSuite level.

The Sampler and Manager in TheadsGroup are equivalent to a Step in SoapUI, as shown below:

Postman is simpler in function and more lightweight in organization, and it is mainly aimed at a single HTTP request. Collection is equivalent to Project, and Collection can create Folders of indefinite levels, and can organize TestSuite by itself. Each Request can be regarded as a TestCase or Step:

 

2.   Supported interface types and test types

In terms of function, Jmeter is the most powerful and can test various types of interfaces. If it does not support it, it can also be extended through plug-ins written on the Internet or by yourself. SoapUI specifically targets two types of HTTP interfaces, and its original intention is to specifically test Soap type interfaces, and does not support interfaces of other protocols. Postman is more lightweight and positioned differently, and can be used to test Rest interfaces.

 

tool	Interface Type	test type
SOAP	Soap、Rest	function, pressure, safety
Jmeter	Rest, Soap, etc. Extensible WebSocket, socket	function, pressure
Postman	Rest	Function

 

3.   Configure different interface types

SoapUI can create Soap Project or Rest Project (but what type of Step is added to the Project is not affected), can add wsdl and wadl resources, and can add Rest or Soap Step in TestCase.

Jmeter can add HTTP, TCP or WebSocket Samplers to the thread group.

Postman only supports Rest interface.

 

4.   Custom variables and the scope of variables

In addition to the variables listed in the table below, each tool has system variables that are not listed.

 

tool	variable type	scope
SOAP	Project, TestSuite, TestCase Properties and Custom Properties	within the respective ranges
	Properties in TestCase	within the entire TestCase
	Data Source, DataGen, etc. in TestCase	within the entire TestCase
	Groovy script definition	look at the definition
Jmeter	User-defined variables in TestPlan	所有Threads Group
	配置元件 - 用户定义的变量	根据元件位置而定
	CSV data set、random variable等	根据元件位置而定
	前置、后置处理器	当前Threads Group
Postman	Environment Variable	当前环境的Collection
	Global Variable	所有Collections
	CSV/JSON datafile	Runner当前的Collection

 

5.  数据源、生成器，进行参数化

 

工具	数据源	生成器	循环
SoapUI	DataSource，数据可来源于文件、目录、数据库、Excel、Grid等	DataGen	DataSource Loop
Jmeter	CSV Data Set Config读取csv文件	Random Variable 计数器	ForEach控制器 循环控制器 While控制器
Postman	Runner中运行时，可加载CSV/JSON文件	无（只能通过脚本）	Runner中的Iteration

 

6.  流程控制

SoapUI：由Conditioinal Goto控制流程，以及Groovy脚本

Jmeter：由Switch控制器、If控制器、随机控制器等一系列控制器实现流程控制，以及Beanshell脚本

Postman：通过JavaScript脚本控制

 

7.  结果解析、展示

 

工具	结果	日志	报告
SoapUI	Project-OverView、TestSuites TestSuite-TestCases TestCase-TestSteps	SoapUI全局多种log TestSuite log TestCase log Request log	Project report TestSuite report TestCase report (PDF/HTML/XML/CSV)
Jmeter	各种监听器	统一的Jmeter log	监听器可导出到文件 并可导出JTL、CSV文件、通过插件可导出HTML（Jmeter3自带）
Postman	Send可查看Request的Response Runner可查看运行的Result	Postman console Chrome DevTools	Request的Response以及Runner的Result均可导出json

 

8.  断言

SoapUI：每个Request可添加Assertion

Jmeter：TestPlan、Threads Group、Sampler均可添加断言

Postman：请求的Tests中可添加断言

 

9.  脚本扩展能力

SoapUI：Groovy脚本

Jmeter：Bean shell（Java）

Postman：JavaScript

 

10. 团队协作

SoapUI：本身一个project是一个xml文件，但是可以通过配置变成一系列文件夹，每个Case、每个Suite均是独立的文件，这样可通过svn/git进行团队协作。支持性较好。

Jmeter：一个TestPlan也是一个jmx（xml）文件，无法分割，但Jmeter有一个合并的功能，允许将多个文件合并在一起。只能每个团队成员自己建立一个TestPlan，分功能块进行测试。最后整理合并。

Postman：有团队协作的功能，需要付费。