[python automation] July PytestAutoApi open source framework study notes (1)

Preface

The content of this article is to learn the relevant knowledge points recorded by the open source framework PytestAutoApi of July Boss for everyone to learn and discuss.

Project address: https://gitee.com/yu_xiao_qi/pytest-auto-api2

Before reading this article, please have an overall study of the framework, and please read the author's README.md file carefully.

The catalog of this article is as follows:

Article directory

ast.literal_eval()

Reference article: https://blog.csdn.net/xili2532/article/details/115393854

The function is similar to eval. The function explanation given in the official document of eval() is: convert the string object into a valid expression to participate in the evaluation operation and return the calculation result. However, eval also has great security risks, such as the user inputting some Malicious strings perform actions. eval will return EOF error when processing an empty string, or grammatical format problem, missing brackets, etc.

So here comes another safe processing method ast.literal_eval. , which will determine whether the content to be calculated is a legal Python type after calculation. If so, the operation will be performed, otherwise the operation will not be performed. This greatly reduces the risk of the system

The difference between os.sep.join and os.path.join

os.sep.joinand os.path.joinare both methods for concatenating file paths, but they work differently.

os.sepIs a string constant that represents the operating system path separator, such as in Windows systems \\and in Unix systems /. The method is to concatenate os.sep.jointhe strings in the list or tuple to form a path string.os.sep

for example:

import os

path_list = ['usr', 'local', 'bin']
path_str = os.sep.join(path_list)

print(path_str)
# Unix系统中输出：usr/local/bin
# Windows系统中输出：usr\local\bin

The method os.path.joinis to automatically concatenate path strings according to different path separators of different operating systems. This way, file paths can be correctly spliced in different operating systems.

for example:

import os

path_str = os.path.join('usr', 'local', 'bin')

print(path_str)
# Unix系统中输出：usr/local/bin
# Windows系统中输出：usr\local\bin

Therefore, if you want to splice paths, it is recommended to use os.path.jointhe method, which can ensure the portability of the code. The method is os.sep.joinmore suitable for use when custom path separators are required in the splicing path.

What does adding an * sign to the def method in Python mean?

In Python function definitions, *args and **kwargs represent receiving a variable number of positional parameters and keyword parameters respectively. When a function is defined, using a single * as a formal parameter prefix indicates that a variable number of positional parameters are received, while a double asterisk ** prefix indicates that a variable number of keyword parameters are received.

When using a single asterisk * as an argument prefix when calling a function, it means splitting an iterable object (such as a list or tuple) into independent positional parameters, while using a double asterisk ** prefix means splitting it into independent positional parameters. A dictionary type object is split into separate keyword arguments.

Therefore, if *args is used in a function definition, it means that the function will receive any number of positional arguments, which will be collected into a tuple; if **kwargs is used in the function definition, it means that the function will receive any number of positional arguments. Keyword arguments, which will be collected into a dictionary. Using both *args and **kwargs in a function definition means that the function will receive any number of positional and keyword arguments.

If you add a single asterisk * before the parameters in the function definition, it means that all positional parameters are collected into a tuple. If you add two asterisks ** before the parameters in the function definition, it means that all keyword parameters are collected into a dictionary. If both * and ** are used, it means that the positional arguments are collected into a tuple, and the keyword arguments are collected into a dictionary.

python @property

In Python, the @property decorator is used to convert a method into a read-only property, allowing callers of the class to call the method as if accessing the property without explicitly calling the method name. Normally, if you need to get the value of an attribute, you need to write a getter method. However, with the @property decorator, the method can be called like other properties of the class, simplifying the code.

Here is an example using the @property decorator:

class Person:
    def __init__(self, name):
        self._name = name
    
    @property
    def name(self):
        return self._name

In the above example, the @property decorator converts the name method into a read-only property. This means that if you want to get the name property of a Person instance, you can simply access it using the name property of the instance object, as follows:

person = Person('Alice')
print(person.name)

This will output Alicethe value of the name attribute of the person instance. Note that there is no need to explicitly call name()the method to get the value.

It should be noted that the @property decorator can only be used for read-only properties. If you need to set the value of a property, you need to use a name.setterdecorator similar to define a setter method.

Delete files in a loop

def del_file(path):
    """删除目录下的文件"""
    list_path = os.listdir(path)
    for i in list_path:
        c_path = os.path.join(path, i)
        if os.path.isdir(c_path):
            del_file(c_path)
        else:
            os.remove(c_path)

pytest hook function

Official website: https://docs.pytest.org/en/6.2.x/_modules/_pytest/hookspec.html#pytest_terminal_summary

pytest_terminal_summary

def pytest_terminal_summary(terminalreporter):
    """
    收集测试结果
    """

    _PASSED = len([i for i in terminalreporter.stats.get('passed', []) if i.when != 'teardown'])
    _ERROR = len([i for i in terminalreporter.stats.get('error', []) if i.when != 'teardown'])
    _FAILED = len([i for i in terminalreporter.stats.get('failed', []) if i.when != 'teardown'])
    _SKIPPED = len([i for i in terminalreporter.stats.get('skipped', []) if i.when != 'teardown'])
    _TOTAL = terminalreporter._numcollected
    _TIMES = time.time() - terminalreporter._sessionstarttime
    INFO.logger.error(f"用例总数: {
      
      _TOTAL}")
    INFO.logger.error(f"异常用例数: {
      
      _ERROR}")
    ERROR.logger.error(f"失败用例数: {
      
      _FAILED}")
    WARNING.logger.warning(f"跳过用例数: {
      
      _SKIPPED}")
    INFO.logger.info("用例执行时长: %.2f" % _TIMES + " s")

    try:
        _RATE = _PASSED / _TOTAL * 100
        INFO.logger.info("用例成功率: %.2f" % _RATE + " %")
    except ZeroDivisionError:
        INFO.logger.info("用例成功率: 0.00 %")

This will be displayed on the console after the use case is executed

insert image description here

pytest_configure

The hook function source code is in \Lib\site-packages\_pytest\hookspec.pythe folder

This is equivalent to setting the execution rules of pytest in pytest.ini, mainly to solve the custom mark warnings information.

Setting it in pytest.ini config.addinivalue_linehas the same effect as setting the mark mark in the conftest.py file.

pytest_collection_modifyitems

Summary: Change the execution order of use cases

Reference: https://blog.csdn.net/mashang_z111/article/details/127266694

[Strictly speaking, in our use case design principle, the use cases should not have a dependent order, so that the meaning of the test cases can be better reflected. (The execution of test cases does not need to be executed in order, but executed immediately)】

pytest_collection_modifyitems 是在用例收集完毕之后被调用，可以用来调整测试用例执行顺序；
它有三个参数，分别是：
 
session：会话对象；
config：配置对象；
items：用例对象列表；改变items里面用例的顺序就可以改变用例的执行顺序了
 
这三个参数分别有不同的作用，都可以拿来单独使用，修改用例执行顺序主要是使用 items 参数【用例执行之前，收集到的测试用例会以元素对象的方式存放在用例对象列表items中】

What is the difference between staticmethod, classmethod and ordinary method?

In Python, functions in a class definition can be defined as three different types of methods: staticmethod, classmethodand ordinary methods (also called instance methods). The differences between these methods are as follows:

Common method:

Ordinary methods are the most common type of methods in classes. These methods can access and manipulate instance properties and methods of the class, and can also use selfthe keyword to refer to the instance on which the method is called. Ordinary methods are typically used to perform certain operations on class instances.

For example:

MyClass:
    def my_method(self, arg1, arg2):
        self.arg1 = arg1
        self.arg2 = arg2
        return self.arg1 + self.arg2

obj = MyClass()
result = obj.my_method(1, 2)
print(result)

In this example, my_methodis a normal method that accesses the instance properties and MyClassof the class and returns their sum.arg1arg2

staticmethod：

staticmethod@staticmethodIs a method decorated by a decorator . This decorator makes the method a static method, that is, the method does not need to access the instance properties and methods of the class, nor the class properties and methods of the class, so there is no need to pass or selfparameters cls. Static methods are often used to implement helper functions associated with a class.

For example:

MyClass:
    @staticmethod
    def my_static_method(arg1, arg2):
        return arg1 + arg2

result = MyClass.my_static_method(1, 2)
print(result)

In this example, my_static_methodit is a static method that does not need to access the instance properties and methods of the class, nor the class properties and methods of the class, so there is no need to pass selfor clsparameters.

classmethod：

classmethod@classmethodIs a method decorated by a decorator . This decorator turns the method into a class method, that is, the method needs to access the class attributes and methods of the class, so clsparameters need to be passed. Class methods are typically used to implement factory methods associated with a class.

For example:

MyClass:
    count = 0

    @classmethod
    def from_count(cls):
        obj = cls()
        obj.count = cls.count
        cls.count += 1
        return obj

obj1 = MyClass.from_count()
obj2 = MyClass.from_count()
print(obj1.count, obj2.count)

In this case, from_counta class method, it needs to access the class attributes of the class count, so it needs to pass clsparameters. This method is used to create MyClassan instance of the class and countset its property to the value of the class's class property count. This method increments the value of the property each time it is called countto create a unique identifier for each new instance.

About caching

my question:

I read the contents of a file directly into python's cache to avoid frequent IO operations. I want to know how big a cache python can have to ensure that the file data I read is not lost.

The Python language has no parameters that directly limit the cache size, so in theory, the size of the Python cache can be increased without limit, but in practice, this depends on the limitations of your computer's hardware and operating system.

Generally speaking, objects created by Python in memory will occupy a certain amount of space, and Python's memory usage is limited by the physical memory size of the computer. Therefore, when you are reading large files, the cache size should be smaller than the available memory size of your computer to avoid out of memory conditions.

Also, be aware that reading the entire file into memory may cause the Python interpreter or program to crash, especially for very large files. Therefore, if you need to process large files, it is better to read them line by line or in chunks instead of reading the entire file into memory. This can effectively reduce memory usage and avoid program crashes.

About safely reading yaml data

Example code:

res = yaml.load(data, Loader=yaml.FullLoader)

This code is the YAML format data that will be read

data is parsed as a Python object.

The function of yaml.load() is to parse the read YAML string into a Python object, and yaml.FullLoader is used here.

yaml.FullLoader is a safe Loader in the yaml library, it can deserialize any Python object, and will not execute any code.

The res variable here will hold the Python object.

It should be noted that this function is deprecated in newer versions of the PyYAML library because of security issues.

The new version of the PyYAML library recommends using the **yaml.safe_load()** function instead of the yaml.load() function to avoid security issues such as Python serialization attacks.

The yaml.safe_load() function only serializes basic Python object types and cannot deserialize any data with Python object types.

At the same time, after Python 3.10, yaml.FullLoader will be marked as dangerous, and it is recommended to use yaml.SafeLoader instead.

enumeration type enum

Reference article: https://blog.csdn.net/yuxuan89814/article/details/128592704

1. enum module

Python's enumeration type is the Enum class that inherits the enum module and defines its own enumeration class. The enumeration elements are equivalent to class variables.

from enum import Enum
class colorEnum(Enum):
    red = 1
    yellow = 2
    blue = 3

The enumeration type is in the form of name=value. The name cannot be repeated, and the value can be repeated, but the alias of the repeated value is the first one.

print(colorEnum.red) #1colorEnum.red
print(colorEnum.gray) #1colorEnum.red
print(colorEnum.red.value) #11

If you want the names in the enumeration classes to be different, you need to introduce unique

from enum import Enum, unique
@unique
class colorEnum(Enum):
    red = 1
    yellow = 2
    blue = 3
    gray=1
print(colorEnum.red)
# ValueError: duplicate values found in <enum 'colorEnum'>: gray -> red

2. Use of enumeration types

from enum import Enum
class colorEnum(Enum):
    red = 1
    yellow = 2
    blue = 3
  
print(colorEnum.red)
print(type(colorEnum.red))
print(colorEnum.red.value)
print(type(colorEnum.red.value))

########### 结果
colorEnum.red
<enum 'colorEnum'>
1
<class 'int'>

Other ways to get enumeration values:

Enumeration variables are in the form of name=value:

You can use value to get the value value or name

You can use the name variable to get the value value or name

print(colorEnum(1))
print(colorEnum(1).value)
print(colorEnum["red"])
print(colorEnum["red"].value)

#### 结果
colorEnum.red
1
colorEnum.red
1

Get all names and values

Use the magic method __members__ to get values and names, the result is a dict array

print(colorEnum.__members__.values())
print(colorEnum.__members__.keys())

########## 
dict_values([<colorEnum.red: 1>, <colorEnum.yellow: 2>, <colorEnum.blue: 3>])
dict_keys(['red', 'yellow', 'blue'])

Practical example:

Determine whether the variable is in our enumeration class, and if so, print out the value:

colors=["red","blue","pink"]
for color in colors:
    if color in colorEnum.__members__.keys():
        print(colorEnum[color].value)
        
########## 
1
3

Mapping function of enumeration value value: name:`_value2member_map_`

from enum import Enum
class colorEnum(Enum):
    red = 1
    yellow = 2
    blue = 3

print(colorEnum._value2member_map_)
###
{
    
    1: <colorEnum.red: 1>, 2: <colorEnum.yellow: 2>, 3: <colorEnum.blue: 3>}

Example of use:

color_nums=[1,2,4]
for color_num in color_nums:
    if color_num in colorEnum._value2member_map_:
        print(colorEnum(color_num).name)
        
######
red
yellow

class colorEnum(Enum):
    red = 1
    yellow = 2
    blue = 3
    gray = 1


for color_num in colorEnum._value2member_map_.keys():
    print(color_num)

print(colorEnum._value2member_map_.keys())
print(type(colorEnum._value2member_map_.keys()))

###########
1
2
3
dict_keys([1, 2, 3])
<class 'dict_keys'>

re.sub function in python

Reference article: https://blog.csdn.net/jackandsnow/article/details/103885422

re is a regular expression, sub is substitute, which means replacement.

re.sub is a relatively complex replacement

re.sub(pattern, repl, string, count=0, flags=0)

Parameters of re.sub: 5 parameters

Parameter 1: pattern

Represents the pattern string in the regular expression.

Parameter 2: repl

It is replacement, which represents the replaced string, which can be a string or a function.

Parameter 3: string

Represents the original string to be processed and replaced

Parameter 4: count

Optional parameter, indicating the maximum number of times to be replaced, and must be a non-negative integer. This parameter defaults to 0, that is, all matches will be replaced;

Parameter 5: flags

Optional parameter, indicating the matching mode used during compilation (such as ignoring case, multi-line mode, etc.), in numeric form, defaulting to 0.

example:

replace only the first two

import re
a = '44444'
b = re.sub('4', '2', a, 2)
print(b) # 22444

Match multiple consecutive Chinese characters

import re
a = '   (rr 我)#1  (d 只是)#1  (p 以)#1  (vi 笑) (v 作答)#1#2#3 (。 。)'
a = re.sub(u"[\u4e00-\u9fa5]+", '*', a) # 匹配多个连续汉字，替换为*
print(a) 
## 输出
   (rr *)#1  (d *)#1  (p *)#1  (vi *) (v *)#1#2#3 (。 。)

Match other symbols besides Chinese characters

# 正则表达式  u"[\u4e00-\u9fa]"  表示所有的汉字  [^...] 表示除了...之外
a = '“设置文件名，怎么样？”'
a = re.sub(u"[\u4e00-\u9fa]", '', a)
print(a)   # 设置文件名怎么样