Web UI automation tool selection survey (seldom)

1. Background

At present, the company's business is relatively stable, but the daily update requirements require a large number of repeated regression tests, which take up part of the time. In order to improve efficiency, UI automation can be used to reduce the manual execution of some personnel to improve efficiency, and it can also be used as a return to the main process function of the daily online environment.

2. What types of projects are suitable for automation

• Requirements do not change frequently: Because the requirements change frequently, the functions of the page will change frequently
• UI is relatively stable: because UI automation is based on UI
• Long project cycle
• A large number of regression testing tasks: a large number of repeated regression testing tasks, continuous iteration, need to return to the old function.
• Smoke test: Conduct a smoke test for the new functions of this iteration (core and backbone functions, about 10%~20%).
• Regression testing: Regression testing of old functionality

3. Automated test scenarios

Scenarios such as:

• Open the website (for example: open the Taobao website)
• Locating elements (for example: locating to the search input box)
• Operational elements (for example: enter autumn clothes in the search box and click Search)
• Simulate page actions (such as: pull down, slide up, etc.)
• Assertion result: compare the expected result with the actual result to judge whether the test is passed.
• generate report

4. UI automation tool selection

4.1 Seldom VS Macaca

compare results:

Compared with Macaca, seldom supports web/app/api testing. It is a full-featured testing framework, and provides scaffolding to quickly create automation projects. It also integrates various methods of test reports and report sending. The most important thing is to provide technical support , to facilitate problem solving. Since Macaca currently has few users and is no longer officially maintained, and the community is no longer active, seldom is currently selected as the web-side test solution.

4.2 seldom framework

4.2.1. Introduction

Seldom is developed based on the unittest unit test framework, and integrates the web/app/api test library, which can perform web UI and APP UI automated testing and interface testing

seldom features

• Supported test types (web/app/api)
• rich assertion
• Generate random test data
• use case dependencies
• Use case classification label
• Support sending (email, DingTalk, Feishu, Qiwei) messages, etc.
• log printing
• cache cache
• command line tool
• Powerful data-driven (JSON/YAML/CSV/EXCEL)
• HTML/XML report
• Failed rerun & screenshot
• Database operation (MySQL/sqlite3/Mongodb)
• Support platform

4.2.2 Installation

 pip install seldom

To experience the latest code at any time, you can use the following command

pip install -U git+https://github.com/defnngj/seldom.git@master

4.2.3 Project creation

seldom -P mypro

Directory Structure:

mypro/
├── test_dir/
│   ├── __init__.py
│   ├── test_web_sample.py
│   ├── test_api_sample.py
├── test_data/
│   ├── data.json
├── reports/
└── confrun.py

• test_dir/Test case directory.
• test_data/Test data file directory.
• reports/Test report directory.
• confrun.pyRun the test case configuration file

4.2.4 Use case creation

Web UIWrite or HTTPinterface automation test according to your own needs

Use case writing rules:

1. A script is a complete scene, from the user login operation to the user exiting the system and closing the browser
2. A script script only verifies one function point, do not try to verify all functions after the user logs in and then log out of the system
3. Try to only do the verification of the forward logic in the function, don't consider too much verification of the reverse logic, there are many cases of reverse logic
4. Try not to create dependencies between scripts
5. Use cases are selected from manual tests, try to choose simple and need to return repeatedly, stable and will not change frequently, priority is given to covering core functions

4.2.5 Running tests

seldom operates in three ways:

• main() method: Use the seldom.main() method in the .py file.
• seldom command: specify the directory & file & class & method to be run through the sedom command.
• Right-click execution of pycharm: This method cannot read the configuration and has serious flaws.
  a. The main() method runs the test

Create test_sample.pya file and use main()the method in the test file as follows:

# test_sample.py
import seldom
from seldom import data


class YouTest(seldom.TestCase):

    def test_case(self):
        """a simple test case """
        self.assertEqual(1+1, 2)

    @data([
        ("case1", "seldom"),
        ("case2", "XTestRunner"),
    ])
    def test_ddt(self, name, search):
        """ ddt case """
        print(f"name: {name}, search_key: {search}")


if __name__ == '__main__':
    # 指定运行其他目录&文件
    seldom.main(path="./")  # 指定当前文件所在目录下面的用例。
    seldom.main(path="./test_dir/")  # 指定当前目录下面的test_dir/ 目录下面的用例。
    seldom.main(path="./test_dir/test_sample.py")  # 指定测试文件中的用例。
    seldom.main(path="D:/seldom_sample/test_dir/test_sample.py")  # 指定文件的绝对路径。
    
    # 运行当前文件中的用例
    seldom.main()  # 默认运行当前文件中所有用例
    seldom.main(case="test_sample")  # 指定当前文件
    seldom.main(case="test_sample.TestCase")  # 指定测试类
    seldom.main(case="test_sample.TestCase.test_case")  # 指定测试用例

    # 使用参数化的用例
    seldom.main(case="test_sample.TestCase.test_ddt")  # 错误用法
    seldom.main(case="test_sample.TestCase.test_ddt_0_case1")  # 正确用例

b.seldom command execution

> cd mypro/  # 进入项目根目录
> seldom -p test_dir  # 运行目录
> seldom -p test_dir/test_sample.py  # 运行文件
> seldom -m test_dir.test_sample       # 运行文件
> seldom -m test_dir.test_sample.SampleTest # 运行 SampleTest 测试类
> seldom -m test_dir.test_sample.SampleTest.test_case # 运行 test_case 测试方法

4.2.6 Test report

seldom generates HTML test reports by default, and automatically creates the reports directory under the running test file

Define test report

seldom.main(path='testadminLoginTest.py',
                title="运营后台登录页面UI测试报告",
                description="admin登录",
                report="adminlogintest.html",
                tester="xxx",
                language='zh-CN'
                )

• report: Configure the report name and path.
• title: The title of the custom report.
• tester: Specify the name of the automated test engineer.
• description: add report information

Access DingTalk Robot

• DingTalk uitest group robots:
  • Signature: xxxx
  • access_token：xxxx

import seldom
from seldom import DingTalk

# ...

if __name__ == '__main__':
    seldom.main()
    ding = DingTalk(
        access_token="xxxx",
        key="",
        app_secret="xxxxx",
        at_mobiles=[13700000000, 13800000000],
        is_at_all=False,
    )
    ding.sender()

Parameter Description:

• access_token: the access_token of the DingTalk robot
• key: If the DingTalk robot security has set keywords, you need to pass in the corresponding keywords.
• app_secret: If the DingTalk robot security is set to sign, you need to pass in the corresponding secret key.
• at_mobiles: A list of mobile phone numbers of @people in DingTalk, such as: [137xxx, 188xxx].
• is_at_all: Whether @ everyone, the default is False, set to True will @ everyone

send email

import seldom
from seldom import SMTP

# ...

if __name__ == '__main__':
    report_path = "/you/path/report.html"
    seldom.main(report=report_path)
    smtp = SMTP(user="[email protected]", password="abc123", host="smtp.126.com")
    smtp.sendmail(to="[email protected]", subject="Email title", attachments=report_path, delete=False)

• subject: email title, default: Seldom Test Report.
• to: Add recipients, multiple recipients are supported: [“[email protected]”, “[email protected]”].
• attachments: set attachments, send HTML test report by default.
• delete: Whether to delete reports & logs. (running automation on the server will generate a report and log every time, manual deletion is more troublesome

4.2.7 Element positioning

seldom provides 8 positioning methods, consistent with Selenium.

• id_
• name
• class_name
• tag
• link_text
• partial_link_text
• css
• xpath

self.type(id_="kw", text="seldom")
self.type(name="wd", text="seldom")
self.type(class_name="s_ipt", text="seldom")
self.type(tag="input", text="seldom")
self.type(xpath="//input[@id='kw']", text="seldom")
self.type(css="#kw", text="seldom")
self.click(link_text="hao123")
self.click(partial_link_text="hao")

Use priority:

ID>Name>CSS>XPath

The highest priority: id, but id sometimes changes, if the id is unchanged, it can be used to locate the
second priority: name priority
: css priority
: xpath

Points to note:

1. If HTML ids are available and unique, then it is the preferred method of locating elements on the page

2. If there is no unique id or name, then use XPath and CSS to locate, XPath selectors are slower

3. Try to let developers add id or name attributes to key elements to reduce unreasonable page elements

4.2.8 Affirmations

# 断言标题是否等于"title"
self.assertTitle("title")

# 断言标题是否包含"title"
self.assertInTitle("title")

# 断言URL是否等于
self.assertUrl("url")

# 断言URL是否包含
self.assertInUrl("url")

# 断言页面包含“text”
self.assertText("text")

# 断言页面不包含“text”
self.assertNotText("text")

# 断言警告是否存在"text" 提示信息
self.assertAlertText("text")

# 断言元素是否存在
self.assertElement(css="#kw")

# 断言元素是否不存在
self.assertNotElement(css="#kwasdfasdfa")

4.2.9 Page Objects design pattern

The design concept of seldom API is to write element operation and element positioning together, and it needs to be used together with poium.

> pip install poium==1.1.6

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# author： HuJun
# datetime： 2023/3/3 16:55
# -*-coding:utf-8-*-
# 用于登录页元素定位
from models import Url
from poium import Page, Element


class login(Page):
    """ 项目用户登录、退出定位元素"""
  

    '用户名输入框'
    username_input_loc = Element(class_name="el-input__inner")
    '密码输入框'
    password_inpput_loc = Element(xpath="/html/body/section/div/div[2]/form/div[4]/div/div/input")
    '登录按钮'
    loginButonClick_loc = Element(class_name="action-group")
    '验证码输入框'
    verificationCode_loc = Element(xpath="/html/body/section/div/div[2]/form/div[7]/div/div[1]/input")
    '退出系统按钮'



    # 操作方法封装
    def username_input(self,key):
        self.username_input_loc.clear()
        self.username_input_loc.send_keys(key)

    def passWord_input(self,key):
        self.password_inpput_loc.clear()
        self.password_inpput_loc.send_keys(key)

    def loginButon_click(self):
        self.loginButonClick_loc.click()

    def verificationCode_input(self,key):
        self.verificationCode_loc.send_keys(key)

    def login_all(self):
        self.username_input_loc.clear()
        self.username_input_loc.send_keys("admin")
        self.password_inpput_loc.clear()
        self.password_inpput_loc.send_keys("ngmmsupermanager")
        self.loginButonClick_loc.click()

Finally : In order to give back to the die-hard fans, I have compiled a complete software testing video learning tutorial for you. If you need it, you can get it for free【保证100%免费】

Software Testing Interview Documentation

We must study to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Ali, Tencent, and Byte, and some Byte bosses have given authoritative answers. Finish this set The interview materials believe that everyone can find a satisfactory job.

How to obtain the full set of information: