Automated testing framework seldom

seldom introduction

seldom is a full-featured automated testing framework based on unittest; it can be used out of the box for automated testing.

seldom features

• Supported test types (web/app/api)
• rich assertion
• Generate random test data
• use case dependencies
• Use case classification label
• Support sending (email, DingTalk, Feishu, Qiwei) messages, etc.
• log printing
• cache cache
• command line tool
• Powerful data-driven (JSON/YAML/CSV/EXCEL)
• HTML/XML report
• Failed rerun & screenshot
• Database operation (MySQL/sqlite3/Mongodb)
• Support platform

 Official documentation: Introduction | seldom documentation

Comparison between seldom and pytest

The author of seldom compares seldom to a computer and pytest to a CPU.

 

Install

pip install seldom

scaffolding creation project 

> seldom -P mypro

 Create test cases

# test_sample.py
import seldom



class SampleTest(seldom.TestCase):

    def test_case(self):
        """a simple test case """
        self.open("http://www.itest.info")
        self.assertInUrl("itest.info")



if __name__ == '__main__':
    seldom.main()
基本规范：

创建测试类YouTest并继承seldom.TestCase类。
创建测试方法test_case, 必须以test开头。

seldom does not need to initialize webdriver, nor does it need to define browser. Only the open method is needed to access the url.

Looking at the source code of the open method, you can see that the driver is defined in seldom and the Chrome browser is called.

(Here, I think the encapsulation should not be so deep, the user should define the driver, and then use the driver to call the method)

 

Operation mode

• main() Method: .py Use seldom.main() the method in the file. Execute with the python command line. python+test file
• seldom Command: sedom Specify the directory & file & class & method to be run through the command.

> cd mypro/  # 进入项目根目录
> seldom -p test_dir  # 运行目录
> seldom -p test_dir/test_sample.py  # 运行文件
> seldom -m test_dir.test_sample       # 运行文件
> seldom -m test_dir.test_sample.SampleTest # 运行 SampleTest 测试类
> seldom -m test_dir.test_sample.SampleTest.test_case # 运行 test_case 测试方法

main parameter

 confrun.py configuration file

confrun.py Used to configure the operating environment. The configuration function  seldom.main() is consistent with the parameters of .

 fail rerun

Set rerun=3, run python on the command line and you can see that it has been executed 3 times, but it cannot be seen that it has been executed 3 times by directly using unittest execution.

seldom.main(rerun=3)

 

 Multi-threaded operation

import seldom
from seldom.utils import threads

#这里有问题，没有打开edge浏览器
class MyTest(seldom.TestCase):

    def test_baidu(self):
        self.open("https://www.baidu.com")
        print("使用chrome运行")
        self.sleep(3)

    def test_bing(self):
        self.open("https://www.bing.com")
        print("使用edge运行")
        self.sleep(4)


@threads(2)  # !!!核心!!!! 设置线程数
def run_case(case: str, browser: str):
    """
    根据传入的case执行用例
    """
    seldom.main(case=case, browser=browser, debug=True)


if __name__ == "__main__":
    # 将两条用例拆分，分别用不同的浏览器执行
    cases = {
        "test_thread_case.MyTest.test_baidu": "chrome",
        "test_thread_case.MyTest.test_bing": "edge"
    }

    for key, value in cases.items():
        run_case(key, value)

start/end

Similar to setup, teardown

start()/end()  

start_class()/end_class()

class TestCase(seldom.TestCase):

    def start(self):
        print("一条测试用例开始")

    def end(self):
        print("一条测试结果")

    def test_search_seldom(self):
        self.open("https://www.baidu.com")
        self.type_enter(id_="kw", text="seldom")

    def test_search_poium(self):
        self.open("https://www.baidu.com")
        self.type_enter(id_="kw", text="poium")

 skip test

You can skip test methods or test classes

decorator

• skip: Unconditionally skip a test.
• skip_if: Skip the test if the condition is true.
• skip_unless: Skip a test unless the condition is true.
• expected_failure: The test case is expected to fail.

class TestCase(seldom.TestCase):

    def start(self):
        print("一条测试用例开始")

    def end(self):
        print("一条测试结果")

    def test_search_seldom(self):
        assert 1==1
    @seldom.skip()
    def test_search_poium(self):
        assert 1==2
if __name__ == '__main__':
    seldom.main(debug=True)

 

One is passed and one is ignored.

Repeat

 @rerun(100)
    def test_search_seldom(self):
        self.open("https://www.baidu.com")
        self.type_enter(id_="kw", text="seldom")

 random test data

class YouTest(seldom.TestCase):

    def test_case(self):
        """a simple test case """
        word = testdata.get_word()
        print(word)

Instead of using the faker library, it is implemented by seldom's own processing method. 

use case dependencies

This comparison method eliminates the need to manually define the order for dependent use cases.

import seldom
from seldom import depend


class TestDepend(seldom.TestCase):
    @depend("test_002")
    def test_003(self):
        print("test_003")
    @depend("test_001")
    def test_002(self):
        print("test_002")

    def test_001(self):
        print("test_001")



if __name__ == '__main__':
    seldom.main(debug=True)

 Use case classification

Tags set to blacklist will not be executed, but tags set to whitelist will be executed.

from seldom import label


class MyTest(seldom.TestCase):

    @label("base")
    def test_label_base(self):
        self.assertEqual(1+1, 2)

    @label("slow")
    def test_label_slow(self):
        self.assertEqual(1, 2)

    def test_no_label(self):
        self.assertEqual(2+3, 5)


if __name__ == '__main__':
    # seldom.main(debug=True, whitelist=["base"])  # whitelist
    seldom.main(debug=True, blacklist=["slow"])    # blacklist

 

seldom API

Seldom API | seldom documentation (seldomqa.github.io)

Page Object

> pip install poium

import seldom
from poium import Page, Element


class BaiduPage(Page):
    """baidu page"""
    search_input = Element(id_="kw")
    search_button = Element(id_="su")


class BaiduTest(seldom.TestCase):
    """Baidu search test case"""

    def test_case(self):
        """
        A simple test
        """
        page = BaiduPage(self.driver, print_log=True)
        page.open("https://www.baidu.com")
        page.search_input.send_keys("seldom")
        page.search_button.click()
        self.assertTitle("seldom_百度搜索")


if __name__ == '__main__':
    seldom.main(browser="chrome")

Send DingTalk

import seldom
from seldom import DingTalk

# ...

if __name__ == '__main__':
    seldom.main()
    ding = DingTalk(
        access_token="690900b5ce6d5d10bb1218b8e64a4e2b55f96a6d116aaf50",
        key="xxxx",
        app_secret="xxxxx",
        at_mobiles=[13700000000, 13800000000],
        is_at_all=False,
    )
    ding.sender()

 log

from seldom.logging import log

log.trace("this is trace info.")
log.info("this is info.")
log.error("this error info.")
log.debug("this debug info.")
log.success("this success info.")
log.warning("this warning info.")

cache

Login token, many use cases will use the login token, then you can use the cache to temporarily store the login token, thereby reducing repeated actions

cache In essence, the data is temporarily recorded through the json file, and there is no expiration time. You need to do the cleanup in place. For example, cleared when the entire use case starts.

data driven

• Support the use of csv, json, yaml files for data drive
• Support the use of decorators class_data(), @data()
• Support the use of interface data @api_data()
• Support for using DDT

data driven | seldom documentation

web project

GitHub - SeldomQA/seldom-web-testing

seldom provides a set of assertion methods for web pages. 

poiumopen in new window  is Page objectsdesign pattern best practice.

API project

Interface automation project based on seldom:

GitHub - SeldomQA/seldom-api-testing: api automation test project based on seldom framework

The real strength of seldom lies in assertions, logging and reporting 

• support assertJSON()
• support assertPath()
• support assertSchema

seldom integrates GenSONopen in new window  , which can help you automatically generate Json schema

database operation

seldom supports sqlite3, MySQL, SQL Server, MongoDB database operations

testing report

Use python to run the file and a test report will be generated. You can turn on the modal if you don't want to generate an HTML report every time you run it debug.

seldom.main(debug=True)

seldom generates HTML test reports by default, and automatically creates directories under the running test files reports.

Define test report

seldom.main(report="./report.html",
            title="百度测试用例",
            tester="tester",
            description="测试环境：windows 10/ chrome")

testing platform

GitHub - SeldomQA/seldom-platform: Based on the seldom test platform