1. Introduction to Allure
The
Allure framework is a flexible, lightweight, and multilingual test reporting tool. In the test, the useful information can be extracted as much as possible.
Allure is developed by the Java language and supports Pytest, JaveScript, PHP, Ruby, etc.
From the perspective of DEV/QA, Allure provides a detailed test report, such as simplifying the statistics of common defects; failed tests can be divided into bugs and interrupted tests; you can also configure logs, steps, fixtures, attachments, timing , execution history; and integration with TMS, BUG management system, Jenkins, etc. Therefore, through the above configuration, all responsible developers and testers can grasp the test information as much as possible.
From a manager's point of view, Allure provides a clear "big picture", a High Level statistical report that includes features covered, where defects are clustered, what the execution timeline looks like, and many other handy things. The modularity and extensibility of Allure ensures that you'll always be able to fine-tune something to make Allure work better for you.
So let's talk about how to make the report display the information we need in more detail, and the integration of Allure and Jenkins.
Installation
(Windows/Mac common installation method)
Download address: https://github.com/allure-framework/allure2/releases, download the zip package of the required version.
Install:
Unzip—> Enter the bin directory—> Run allure.bat
to add the bin directory to the Path environment variable
Cooperate with pytest, use allure2 to generate a more beautiful test report: pip install allure-pytest
running method
Collect results during test execution
pytest [test file] -s -q --alluredir=./result/ --clean-alluredir -s
: means to print out the successful case log
-q: if it follows the file execution path, it means only the file that needs to be executed-
-alluredir: Specify the path to store test results (if the directory does not exist, it will be created)
--clean-alluredir: Clear historical result data
View test reports
Method 1: Used to display the results externally after local rendering
allure serve ./result/Method
2: for rendering and viewing results locally
# Generate report
allure generate ./result/ -o ./report/ --clean # Note: add --clean to the coverage path
# Open the report
allure open -h 127.0.0.1 -p 8883 ./report/
Note: the index.html in the /report/ directory is the final result page, but you cannot see the actual report content if you open this file directly through the browser , this is because the actual report content needs to be rendered by allure before it can be seen.
2. Common features of Allure
Scenario:
Expect to see test functions, sub-functions or scenarios, test steps, including test additional information in the report.
solve:
import allure
@allure.feature('feature name')
@allure.story('sub-feature name')
@allure.title('test case name')
@allure.step('step details')
@allure.description(' Test case description')
@allure.attach('Specific text information'): Additional information is required, which can be data, text, pictures, videos, and web pages.
If you only test the login function, you can add restrictions and filters, such as: pytest file name --allure_features 'cart features' --allure_stories 'add to cart'
The relationship between @alllure.feature() and @allure.store()
feature is equivalent to a large function or module. Classify the case into a feature and display it in the behaviors in the report, which is equivalent to testsuite.
story is equivalent to a branch function/module, belongs to the structure under feature, and is displayed in features in the report, which is equivalent to testcase.
feature and story are similar to parent-child relationship.
The difference between @allure.step() and with allure.step()
Each step in the test process is generally placed in a specific logic method.
It can be placed in the key step and displayed in the report.
In App and Web automated testing, it is recommended that each switch to a new page be regarded as a step.
Usage:
@allure.step(): can only be placed on a class or method in the form of a decorator.
with allure.step(): can be placed in the test case method, but the code of the test step needs to be included in this statement.
Prioritize test cases
Scenario:
Usually tests include smoke test, regression test, online verification test, etc., so they need to be executed according to the level of importance. For example, the main process and important modules must be run once when going online.
solve:
Mark description by appending pytest.mark Mark description
by allure.feature, allure.story Mark test
case level directly by allure.severity Classify
the test case level according to the importance of the test case. If no level is specified, the default is NORMAL level:
BLOCKER: Blocking defects (unrealized function, unable to proceed to the next step)
CRITICAL: Serious defects (missing function points)
NORMAL: General defects (boundary cases, format errors)
MINOR: Minor defects (interface errors do not meet ui requirements)
TRIVIAL: Minor defects (There is no prompt for mandatory items, or the prompt is not standardized)
Steps:
Add above methods, functions and classes: @allure.severity(allure.severity_level.TRIVIAL)
to specify the use case of the corresponding level: pytest -s -v file name --allure-severities normal, critical
Add content to the Allure test report (picture , attachments, text, screenshots, HTML, etc.)
scenarios:
Front-end automated testing often requires additional pictures or html, such as taking screenshots at the right place and at the right time.
solve:
@allure.attach() displays a number of different types of provided attachments that can supplement a test, step or test result.
step:
Attach a web page in the test report:
Format: allure.attach(body(content), name, attachment_typeextension)
Example: allure.attach('<head>/head><body>Homepage</body>', 'This is an error page Result information', allure.attachment_type.HTML)
to attach a picture in the test report:
Format: allure.attach.file(source, name, attachment_type, extension)
Example: allure.attach.file("./result/b.png ", attachment_type=allure.attachment_type.PNG)
integrated test management system
@allure.link(), @allure.issue(), @allure.testcase() is mainly to integrate Allure report and test management system, which can be more quickly Jump to company internal address.
First look at the source code of the three decorators:
def link(url, link_type=LinkType.LINK, name=None):
return safely(plugin_manager.hook.decorate_as_link(url=url, link_type=link_type, name=name))
def issue(url, name=None):
return link(url, link_type=LinkType.ISSUE, name=name)
def testcase(url, name=None):
return link(url, link_type=LinkType.TEST_CASE, name=name)summary
issue() and testcase() actually call link(), but the link_type is different.
Mandatory parameter url: the link to jump to.
Optional parameter name: The name displayed in the Allure report, if not passed, the complete link will be displayed (it is recommended to pass, otherwise the readability is not high).
It can be understood as: the three methods are the same, we all provide jump links and names, but the types of links are different, and the final displayed styles are different (different types, different styles).
Just @allure.link() if you like.
The reason for the appearance of three decorators is to better classify links (visit links, bug links, test case links).
code example
import allure
TEST_CASE_LINK = 'https://github.com/qameta/allure-integrations/issues/8#issuecomment-268313637'
@allure.link('https://www.youtube.com/watch?v=4YYzUTYZRMU')
def test_with_link():
pass
@allure.link('https://www.youtube.com/watch?v=Su5p2TqZxKU', name='点击我看一看youtube吧')
def test_with_named_link():
pass
@allure.issue('140', 'bug issue链接')
def test_with_issue_link():
pass
@allure.testcase(TEST_CASE_LINK, '测试用例地址')
def test_with_testcase_link():
passRun the results, view the Allure report
1) The style when @allure.link() does not pass the name parameter
3. Example of interface automation testing framework
Complete project: GitHub - juno3550/InterfaceAutoTestWithPytest: Interface automation testing framework: Pytest+Allure+Excel data-driven
Example of test method
1 import pytest
2 import allure
3 import logging
4 from util.assert_util import assert_keyword
5 from util.request_util import api_request
6 from util.global_var import *
7 from util.excel_util import excel_util
8
9
10 register_test_data = excel_util.get_sheet_data("注册")
11 login_test_data = excel_util.get_sheet_data("登录")
12
13
14 @allure.feature("登录模块")
15 @pytest.mark.dependency(name="TestLoginModule")
16 class TestLoginModule:
17
18 @allure.story("注册功能")
19 @allure.title('用户注册') # 指定测试用例标题，默认是函数名
20 @allure.description('通过接口进行用户注册') # 添加测试用例描述
21 @allure.severity(allure.severity_level.BLOCKER) # 阻塞级别
22 @pytest.mark.run(order=1)
23 @pytest.mark.parametrize('case_data', register_test_data)
24 def test_register(self, case_data):
25 with allure.step("读取请求数据，调用接口"):
26 logging.info("接口用例数据：%s" % case_data)
27 response = api_request(case_data[API_IP], case_data[API_URI], case_data[REQUEST_METHOD],
28 case_data[API_REQUEST_DATA], case_data[RESPONSE_EXTRACT_VAR],
29 case_data[REQUEST_HEADER], case_data[REQUEST_COOKIE])
30 with allure.step("获取响应数据，进行断言"):
31 assert_keyword(response, case_data[RESPONSE_ASSERT_KEYWORD])
32
33 @allure.story("登录功能")
34 @allure.title('用户登录') # 指定测试用例标题，默认是函数名
35 @allure.description('通过接口进行用户登录') # 添加测试用例描述
36 @allure.severity(allure.severity_level.BLOCKER) # 阻塞级别
37 @pytest.mark.run(order=2)
38 @pytest.mark.parametrize('case_data', login_test_data)
39 def test_login(self, case_data):
40 with allure.step("读取请求数据，调用接口"):
41 logging.info("接口用例数据：%s" % case_data)
42 response = api_request(case_data[API_IP], case_data[API_URI], case_data[REQUEST_METHOD],
43 case_data[API_REQUEST_DATA], case_data[RESPONSE_EXTRACT_VAR],
44 case_data[REQUEST_HEADER], case_data[REQUEST_COOKIE])
45 with allure.step("获取响应数据，进行断言"):
46 assert_keyword(response, case_data[RESPONSE_ASSERT_KEYWORD])
47
48
49 if __name__ == "__main__":
50 test_dir = os.path.dirname(__file__)
51 pytest.main(['-s', '-q', test_dir, '--alluredir', '../test_result/', "--clean-alluredir"])
52 os.system('allure generate ../test_result/ -o ../test_report/ --clean')
53 os.system('allure open -h 127.0.0.1 -p 8881 ../test_report/')Test data example
Example of Allure report results
Friends who are doing the test can come in and communicate. The group has compiled a lot of learning materials and interview questions, project resumes, etc....