Idea analysis and technical points for building an interface automated testing framework

background

Why do automated testing?

Save testing time. Scenarios such as regression testing after the project reaches a certain size require a lot of time and manpower.

Why do we need to do interface automation testing?

Compared with UI, the interface is more stable, and it is more convenient to implement automation. After the code is formed, use cases and code maintenance are also more convenient.

1. The process and scenarios of disassembling interface testing

1. Analyze the contents of the interface request

• l Request method method
• l Request address url
• l Environment configuration ip
• l Pre-operation
• l Request Body (data type)
• l Status code
• l Response body
• l assertion
• l Post-operation

2.  Two interface test scenarios

1. Verify a single interface

Checkpoints such as the data type/length/null value of the interface field are verified. In this scenario, a large number of different data requests the same interface, which is suitable for data-driven (DDT). This scenario mainly solves the problem of batch reading of use case data, loading DDT, and recording execution. The result (using exception handling) is three problems.

1. Test the business flow of the interface

This scenario will involve multiple interfaces and data insertion and cleaning, etc.

1Analyze business relationships

You can start with flow charts, 4+1 views, design documents, etc.

2 carding

Example: Start indicator

Conditions for starting the indicator. The indicator should exist (delete_flag=0) and the indicator status is inactive.

To ensure the independence (decoupling) of use cases, the testing process should be:

1Create a new indicator and return the indicator id

2 Request the indicator startup interface to start the newly created indicator; the assertion interface returns success/the indicator status in the database is refreshed to active; since the indicator startup process and platform construction tasks are time-consuming, a wait needs to be set here.

3. Data cleaning, request the indicator stop interface to stop the indicator task, and delete the data related to this indicator (indicator table/indicator expression table)

2. Analyze interface automated testing activities

1. Interface communication--preparation for writing use cases

Interface documentation

Postman (flexibly use the function of postman to generate code)

Refer to the documentation and use postman to request and open the interface.

2. Management and organization of data related to the testing process

1. Use case writing

Use case data use case number, use case name, url, method, input parameters, expected results

2. Format of use case file, organization method excel/python file/yaml

1 Use excel tables to write use cases, pay attention to the design of the use case table fields

Advantages: simple maintenance, easy modification, no programming required

Disadvantages: The program needs to be supported by relevant modules (openpyxl/xlrd, etc.) before it can be read, and relevant codes need to be written.

example:

 Suitable for single interface verification to be data driven

Suitable for business scenario testing, with regular expressions and reflection

2 Use python files to write use cases

Writing use case data using object-oriented and inherited methods

Advantages: python can be read directly, no middle layer is needed, and can be written directly in the development environment

Disadvantages: Writers need to know Python syntax, understand object-oriented and inheritance, etc. Cross-language is more troublesome

example

class Case:
    #此处可使用类属性放置所有用例实例共有的属性，如IP，但是IP这种环境配置还是建议放在config里，不然用例改环境太麻烦
    host_ip=""
    def __init__(self,num,name,url,method,req_body,exp_result):
        self.num=""
        self.name=""
        self.url=""
        self.method=""
        self.req_body=""
        self.exp_result=""
#将用例属性进行实例化传入，因为整个框架设计数据传递主要使用json，所以建议所有字符串都用双引号
#实例化用例后导入到其他模块使用就可以了
case_01=Case("01","case_01","/abc","post","{"user":"001"}","{"status":"success"}")
#多个用例时，可使用list传递，适用DDT
case_02=Case("01","case_01","/abc","post","{"user":"001"}","{"status":"success"}")
case_03=Case("01","case_01","/abc","post","{"user":"001"}","{"status":"success"}")
case_box=[case_01,case_02,case_03]
#当然，也可以写个循环生成用例，case_box.append()循环往里加

class New_case(Case):

    def __init__(self,num,name,url,method,req_body,exp_result,exp_code): 　　　　 super().__init__(num,name,url,method,req_body,exp_result) 　　　　 self.exp_code = exp_code
#使用继承可以方便添加新的用例属性

3 Use yaml files to write use cases

Advantages: Strong cross-language advantages

Disadvantages: Writers need to know YAML syntax and need to use third-party module support to read data

example

# yaml语法
# key: {key1: value1,key2: value2} 注意：冒号后有空格，字符串建议使用双引号
case_data_workbook:
  workbook: "../data_of_cases/case2.xlsx"
  worksheet: "Sheet1"

# 用例编写
case_01:
  num: "01"
  name: "case_01"
  url: "/abc/o1"
  req_body : '{"credentials": "123456","account":"admin"}'

3. Environment configuration config

1 Use python classes to write configuration files

The advantages and disadvantages are the same as above, let’s look directly at the example

"""
建立一个类
配置项用类属性来写，方便其他模块导入使用
路径可以使用pathlib拼接

"""


class CONFIG:　　#环境host
    HOST = "http://www.xxx.com:8108"　　#用例excel文件
    WORKBOOK = "../data_of_cases/case2.xlsx"　　#用例sheet
    WORKSHEET = "Sheet1"　　#日志输出路径
    LOG_PATH = "../test_logs/runlog_.text"　　#打印日志等级
    LOG_LEVEL = "ERROR"　　#测试报告输出路径
    REPORT_PATH = "../test_report/report.html"

2 Use yaml to write configuration files

The advantages and disadvantages are the same as above. Let’s look at the examples directly.

#使用yaml作为配置文件，比较通用，跨语言有优势
#用例存储文件路径，及sheet页名
#建议采用相对路径

case_data_workbook:
  workbook: "../data_of_cases/case2.xlsx"
  worksheet: "Sheet1"

#log日志输出位置
#打印日志的等级
#日志切分   可按时间切分，也可按大小，如 1 MB切分

log_path: "../test_logs/runlog.log"
log_level: "ERROR"
log_rotation: "12:00"

#测试报告输出位置

report_path: "../test_report/report.html"

#后台域名，结合环境
host: "http://www.xxx.com:8108"
#前台域名
front_host: "http://www.xxx.com:8107"

3 Requirements during the testing process

• frame
• Request method
• Read use case data
• Read environment configuration
• Data preprocessing and cleaning
• Execution result record
• Test report output
• ...

3.  Write the modules and packages required for the automation framework, and technical points

1 frame:

unittest

Advantages: python comes with it and is easier to use.

Disadvantages: The degree of freedom of customization is lower than that of pytest

Regarding the use and rules of unittest, please refer to the great master’s blog  unittest - Duanlang Kuangdao Reminiscing about Youth - Blog Garden (cnblogs.com)

Pytest

Advantages: High degree of freedom in customization, compatible with unittest, and provides a large number of decorators

Disadvantages: Need to download to use

Regarding the use and rules of pytest, please refer to the master's blog  pytest - The Mad Sword of Duanlang Reminiscing about Youth - Blog Garden (cnblogs.com)

2Request method request

3 Read use case data openpyxl  python-operate openpyxl in excel - Duanlang Kuangdao recalls youth - Blog Park (cnblogs.com)

4 Read the environment configuration yaml/python file, see the previous article for details

5Loggingloguru/logging

loging

Advantages: You can write functions completely according to your own needs

Disadvantages: The functions are basic and there are many areas that need to be developed.

Regarding the use and content of logging, you can refer to the master's blog python logging module (detailed analysis)_Tianjian Humalingyueniao's blog-CSDN blog

window

Advantages: Highly packaged, worry-free

Disadvantages: You can use whatever is given to you, but it already meets your needs.

Regarding the use and content of loguru, you can refer to the master's blog Loguru - the most powerful Python logger - Python practical guide (pythondict.com)

6MySQL data processing

There is nothing to say about this. pymysql is easy to use. What needs to be paid attention to is the database transaction conn.commit()

Basic usage

• 1Connect to the database
  • conn = pymysql.connect() gets the connection object
• 2 Get the cursor
  • cursor = conn.cursor()
• 3Use sql statement
  • cursor.excute(sql statement)
• 4Get query results
  • cursor.fetchall(),,, tuples within tuples, a piece of data is an element
  • cursor.fetchone(),,, take out the tuple
• Initialize a cursor before each query
  • The cursorclass parameter in connect is equivalent to the cursor parameter in conn.cursor, which reads out a list of nested dictionary structure data.
  • conn = pymysql.connect(cursorclass=pymysql.cursors.DictCursor)[Image upload failed...(image-d27d50-1686663646198)]

pymysql - Duanlang Kuangdao recalls youth - Blog Garden (cnblogs.com)

7 test report

unittestreport

Applicable to unittest  1. Installation and introduction - unittestreport usage documentation

allure

Applicable to pytest  Pytest testing framework (5): pytest + allure generates test reports - test development notes - Blog Garden (cnblogs.com)

8 technical points

python functions, object-oriented, inheritance, reflection, json, regular expressions, encapsulation, etc., sql syntax, unittest framework, requests, decorators, etc.

4. Framework directory structure

Let’s take a look at the directory structure of the entire project

Use unittest framework excel table to manage use case yaml configuration files

├─common                             #公共包
│ │ tool_of_log.py                   　　#日志模块
│ │ tool_of_mysql.py              　　   #数据库模块
│ │ tool_of_read_config.py     　　      #读取配置模块
│ │ tool_of_read_excel.py      　　      #表格数据处理
│ │ tool_of_requests.py          　　    #通用请求方法
│ │ __init__.py
│ 
├─data_of_cases                     #用例数据包
│ │ case1.xlsx                         　　#用例表格
│ │ case2.xlsx
│
├─data_of_config                    #配置文件包
│ │ config.py                            #python类型的配置文件
│ │ config.yaml                          #yaml类型配置文件
│
├─run                               #测试框架主入口
│ │ main.py                             #主入口
│ │ __init__.py
│
├─setup　　　　　　　　　　       #前置处理包，一些前置处理可以单抽出来，如获取cookie
│ │ 000345.png
│ │ new_account.py
│ │ put_product.py
│ │ uuid_imagecode.py
│ │ __init__.py
│
├─test_logs                               #测试日志包
│ │ runlog.log                            　　#测试日志文件
│
├─test_mainclass                          #测试主类
│ │ test_cla_and_meth.py                　　  #被测类和被测方法
│ │ test_new_account.py                      #场景测试 新增用户
│ │ test_product_upload.py　                 #场景测试 上传产品
│ │ __init__.py
│
└─test_report                                #测试报告
│ │history.json                                  #测试报告相关json文件，自动生成的
│ │report_2022-09-06_23-57-26.html               #测试报告文件

5. Code examples

common

Log module

"""
记录日志模块
sink为日志输出目录
level为写入日志等级
建议日志使用log文件记录
"""

from loguru import logger
from common.tool_of_read_config import read_config_yaml

# 这里是读取配置文件的日志输出位置
config = read_config_yaml("../data_of_config/config.yaml")
logger.add(sink=config["log_path"], level="ERROR", encoding="utf-8")

Database module

"""
连接数据库模块
提供了三个方法
1游标的初始化
2执行查询sql语句并销毁游标，返回查询到的数据
3关闭数据库连接
"""
import pymysql


class Tool_of_mysql:

    def __init__(self, host, port, user, password, dbname):
        # connect中的cursorclass参数,相当于conn.cursor中的cursor参数,读出一个列表嵌套字典结构数据
        self.conn = pymysql.connect(host=host, port=port, user=user, password=password, db=dbname,cursorclass=pymysql.cursors.DictCursor)
        self.cursor = self.conn.cursor()

    def cursor_init(self):
        # 执行一次SQL，游标会销毁，所以提供游标初始化方法
        self.cursor = self.conn.cursor()

    def run_sql(self, sql):
        # 这里建议只使用select语句
        self.cursor.execute(sql)
        # 提交事务
        self.conn.commit()

        db_data = self.cursor.fetchone()

        self.cursor.close()
        # self.conn.close()
        return db_data

    def mysql_close(self):
        self.conn.close()


if __name__ == '__main__':
    mon = Tool_of_mysql(host="47.113.xxx.81", port=3306, user="mon", password="mon123", dbname="yami_sops")
    # mon.cursor_init()

    data = lemon.run_sql("select user_phone,mobile_code from tz_sms_log limit 5")
    print(data)

Read configuration file module

"""
读取yaml配置文件模块
"""
import yaml


def read_config_yaml(filepath):
    with open(filepath, encoding="utf-8") as f:
        config_data = yaml.safe_load(f)
    return config_data


if __name__ == '__main__':
    print(read_config_yaml("../data_of_config/config.yaml"))

Table data processing module

"""
读取excel表格的模块
"""

import openpyxl
from openpyxl.worksheet.worksheet import Worksheet


def read_xlsx_tool(filename, sheetname):
    workbook = openpyxl.load_workbook(filename)
    # 这里做了类型注解，方便ide给出代码联想提示
    sheet: Worksheet = workbook[sheetname]

    data = list(sheet.values)

    title = data[0]

    lines = data[1:]

    cases = [dict(zip(title, row)) for row in lines] #循环 并通过zip组装字段名和字段值并放进一个dict里
    # for row in lines:
    #     cases.append(dict(zip(title, row)))

    return cases

if __name__ == '__main__':

    s = read_xlsx_tool("../data_of_cases/case2.xlsx", "Sheet2")
    print(type(s))
    print(s[0])
    print(type(s[0]))

General request method, this may not be used

"""
封装了请求方法的模块
默认请求方法post
其实该模块可以不封装，直接写代码并不是很多，处理比较方便，封装后加了一层导入
"""


import requests


def request_api(method='post', url=None, headers=None, json_data=None, body_data=None):
    res = requests.request(method=method, url=url, headers=headers, json=json_data, data=body_data)

    return res

data_of_config

yaml environment configuration file

#使用yaml作为配置文件，比较通用，跨语言有优势
#用例存储文件路径，及sheet页名
#建议采用相对路径

case_data_workbook:
  workbook: "../data_of_cases/case2.xlsx"
  worksheet: "Sheet1"

#log日志输出位置
#打印日志的等级
#日志切分   可按时间切分，也可按大小，如 1 MB切分

log_path: "../test_logs/runlog.log"
log_level: "ERROR"
log_rotation: "12:00"

#测试报告输出位置

report_path: "../test_report/report.html"

#后台域名，结合环境
host: "http://www.xxx.com:8108"
#前台域名
front_host: "http://www.xxx.com:8107"

run

Test framework main entrance

"""
运行测试的主模块，同时生成日志，和测试报告
"""
import unittest
import time
import unittestreport

suit = unittest.defaultTestLoader.discover("../test_mainclass")

runner = unittestreport.TestRunner(suit,
                                   filename=f"report_{time.strftime('%Y-%m-%d_%H-%M-%S')}.html",
                                   report_dir="../test_report",
                                   title="迭代10_接口自动化测试报告",
                                   tester="woody",
                                   desc="XX项目迭代10------",
                                   templates=2)
#文件名拼接时间，每次运行都会生成一个报告文件
runner.run()

test_logs

Generated test log

It is recommended to install the plug-in ideolog, pycharm can directly highlight the log

2022-09-06 23:57:26.096 | ERROR    | test_cla_and_meth:test_:36 - 用例---case_004--->执行失败   ====>{'code': 200, 'message': '登录成功', 'token': 'good-token'} != {'code': 200, 'message': '登录失败', 'token': 'none'}
- {'code': 200, 'message': '登录成功', 'token': 'good-token'}
?                             ^^             ^ ^^^^^^ -

+ {'code': 200, 'message': '登录失败', 'token': 'none'}
?                             ^^             ^ ^

2022-09-06 23:57:26.107 | ERROR    | test_cla_and_meth:test_:36 - 用例---case_005--->执行失败   ====>{'code': 200, 'message': '登录成功', 'token': 'good-token'} != {'code': 200, 'message': '登录失败', 'token': 'none'}
- {'code': 200, 'message': '登录成功', 'token': 'good-token'}
?                             ^^             ^ ^^^^^^ -

+ {'code': 200, 'message': '登录失败', 'token': 'none'}
?                             ^^             ^ ^

2022-09-19 23:53:30.900 | ERROR    | test_cla_and_meth:test_:39 - 用例---case_001--->执行失败   ====>'{"code":200,"message":"登录成功","token":"good-token"}' != <Request [post]>

test_mainclass

Tested class and tested method

"""
编写测试类和测试方法
使用unittest框架，导入ddt和list_data两个装饰器，分别装饰测试类和测试方法
导入公共层表格读取模块，读取用例数据
导入公共层配置文件读取模块，读取配置文件
导入公共层日志记录器
导入被测方法/函数
导入json模块处理数据
"""

import unittest

import requests
from unittestreport import ddt, list_data  #特别注意！使用的ddt list_ddt是unittestreport这个模块提供的，不是我们常说的ddt模块 这里看五、数据驱动的使用 - unittestreport 使用文档 功能十分强大！

from common.tool_of_read_config import read_config_yaml
from common.tool_of_read_excel import read_xlsx_tool
from common.tool_of_log import logger
import json

config = read_config_yaml("../data_of_config/config.yaml")
host = config["host"]
cases = read_xlsx_tool(filename=config["case_data_workbook"]["workbook"],
                       sheetname=config["case_data_workbook"]["worksheet"])


# cases = read_xlsx_tool()

@ddt
class Test_cases(unittest.TestCase):
    @list_data(cases[0:1])  # 这里必须传入一个列表，所以定位单个用例，不能直接用索引，要用切片[{}],切出一个只有一个元素的列表
    def test_(self, case):
        # 注意这里的“json=“参数，表格读出来是字符串，要转成Json才能做请求入参
        # 这里的url是拼接的
        response = requests.request(method=case["method"], url=host + case["url"], json=json.loads(case["入参"]))

        # 这里使用try尝试取结果，返回类型不同使用不同的方法
        try:
            act_res = response.json()
        except Exception as e:

            res = response.text
            act_res = {"msg": res}

        expect = json.loads(case["预期结果"])

        # try进行断言是为了写日志，再手动抛出异常
        for k, v in expect.items():
            try:
                self.assertEqual(v, act_res[k])
            except AssertionError as e:
                logger.error(f'{e}')
                raise e
            # self.assertIn(case["预期结果"], res)
        # print(res)
        # expect_res = json.loads(case["预期结果"])  # 转成字典
        # try:
        #
        #     self.assertEqual(expect_res, res)
        # except AssertionError as e:
        #     # logger.error(f"用例---{case['用例名']}--->执行失败")
        #     logger.error(f"用例---{case['用例名']}--->执行失败   ====>{e}")
        #     raise e

Test business scenarios using reflection and regular expression matching

"""
编写测试类和测试方法
使用unittest框架，导入ddt和list_data两个装饰器，分别装饰测试类和测试方法
导入公共层表格读取模块，读取用例数据
导入公共层配置文件读取模块，读取配置文件
导入公共层日志记录器
导入被测方法/函数
导入json模块处理数据
"""
import re
import unittest

import requests
from unittestreport import ddt, list_data

from common.tool_of_read_config import read_config_yaml
from common.tool_of_read_excel import read_xlsx_tool
from common.tool_of_log import logger
import json
import time
from setup.put_product import login, img_put

config = read_config_yaml("../data_of_config/config.yaml")
host = config["host"]
cases = read_xlsx_tool(filename=config["case_data_workbook"]["workbook"],
                       sheetname="Sheet2")


# cases = read_xlsx_tool()

@ddt
class Test_cases(unittest.TestCase):
    def setUp(self) -> None:
        self.token = login()['access_token']
        self.img_id = img_put(self.token, "../setup/000345.png")
        print(self.token, self.img_id)
        self.time = f"{time.time()}"

    @list_data(cases[0:1])  # 这里必须传入一个列表，所以定位单个用例，不能直接用索引，要用切片[{}],切出一个只有一个元素的列表
    def test_(self, case):

        data = ""
        header = {"Authorization": f"bearer{self.token}"}
        # 法1：这里先使用replace替换，注意，相同的，一行代码全部会替换掉
        # data = case["入参"].replace("#time#", f"{time.time()}")
        # data = case["入参"].replace("#img_id#", self.img_id)

        # 法2：使用正则表达式替换，搜索出用#——#标记的字段，切掉#后，用反射读取类属性，然后使用replace方法替换
        flag = re.finditer("#(.+?)#", case["入参"])# 正则搜索匹配后返回一个迭代器 ，元素是"#___#"
        for i in flag:
            old = i.group()  # 用group方法从迭代器取值
            old_ = old.strip("#") # 切掉两头的#号，剩下的就是"token",这种用例入参里的某个键
            new = getattr(self, old_)# 使用反射，getattr拿到属性的值并用变量保存起来
            data = case["入参"].replace(old_, new)# 使用replace方法把属性再替换到入参字符串里
        # 注意这里的“json=“参数，表格读出来是字符串，要转成字典才能做请求入参
        data = json.loads(data)

        # 这里的url是拼接的
        response = requests.request(method=case["method"], url=host + case["url"], json=data, headers=header)

        # response = product_put(self.token,self.img_id)

        # 这里使用try尝试取结果，返回类型不同使用不同的方法
        try:
            act_res = response.json()
        except Exception as e:

            res = response.text
            act_res = {"msg": res}
        print(act_res)
        expect = json.loads(case["预期结果"])

        # try进行断言是为了写日志，再手动抛出异常
        for k, v in expect.items():
            try:
                self.assertEqual(v, act_res[k])
            except AssertionError as e:
                logger.error(f'{e}')
                raise e
            # self.assertIn(case["预期结果"], res)
        # print(res)
        # expect_res = json.loads(case["预期结果"])  # 转成字典
        # try:
        #
        #     self.assertEqual(expect_res, res)
        # except AssertionError as e:
        #     # logger.error(f"用例---{case['用例名']}--->执行失败")
        #     logger.error(f"用例---{case['用例名']}--->执行失败   ====>{e}")
        #     raise e

Thoughts on the application of reflection in interface automation

The previous code shows that reflection is very useful, because the object."attribute" cannot be adjusted with a string, so reflection must be used. hasattr determines whether there is one, and getattr can get the attribute value. If it is a method, use After saving the variable, add () to call it, and setattr to set the new attribute.

What if we put reflection into UI automation?

Encapsulate high-frequency UI operations, such as clicks, inputs, etc.

class Action:　　　　def __init__():　　'''　　初始化浏览器对象等　　'''　　def mouse_click(元素定位: string):
        '''
        例，将鼠标点击的操作，封装，传入元素定位，如id或者xpath，进行鼠标点击
        '''
        pass
    
    def keyboard_input(元素定位: string ，输入内容: string ):　　　　　'''　　　　　键盘输入操作需要定位元素并有输入的内容　　　　　先定位  v[0]　　　　　输入    v[1]　　　　　'''　　　　　　
        pass

The format of the input parameters we pass in can be

{
"mouse_click":元素定位，
"keyboard_input":[元素定位,输入内容],
...
}

Arrange our ui operations into json and pass it in, and fetch and operate the ui through a loop

action = Action()for k,v in case['ui操作'].items:
    move = getattr(action,k)　  if type(v) is list:
    　　move(*v)  # 调用解包，传入一个序列打散后当位置参数传进去　　else:　　　　 move(v)

I have done relatively little research on UI automation, but I recommend taking a look at Playwright

Basic use of Python Playwright (detailed steps) - Nuggets (juejin.cn)

Installation | Playwright Python

How does Python crawl? Play with the new generation of crawler tool Playwright! - Zhihu(zhihu.com)

Compared with selenium, playwright has good execution efficiency, powerful script recording capabilities, and lower code difficulty.

test_report

testing report

Finally: The following is the supporting learning materials, which should be the most comprehensive and complete preparation warehouse for friends who do [software testing]. This warehouse has also accompanied me through the most difficult journey. I hope it can also help you! [100% free to receive without any tricks]

Software testing interview applet

A software test question bank that has been used by millions of people! ! ! Who is who knows! ! ! The most comprehensive interview test mini program on the Internet, you can use your mobile phone to answer questions, take the subway, bus, and roll it up!

Covers the following interview question sections:

1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux

6. Web, app, interface automation, 7. Performance testing, 8. Programming basics, 9. HR interview questions, 10. Open test questions, 11. Security testing, 12. Computer basics

  How to obtain the full set of information: Click on the small card below to get it yourself