Table of contents
Read out the script data and use regular expressions (re library) to match key data for modification
Read out the script data, use BeautifulSoup's xml parsing function to parse and modify
Use string.Template character replacement
background introduction
When we do performance tuning, we often need to adjust the parameters of the thread group according to the actual pressure test, such as the number of cycles, the number of threads, and the start time of all threads.
If it is on a Linux machine, it is unavoidable to open the graphics page on the local machine for modification, and then finally pass it to the stress test machine. In order to solve this business pain point, I use Python to write a tool that can directly modify the Jmeter basic stress test parameters
. The script can modify the number of thread groups, the number of cycles, and the time it takes for all thread groups to start in the jmx script.
Implementation ideas
When I first started writing this script, I thought of two ideas:
Read out the script data and use regular expressions (re library) to match key data for modification
Advantages: data can be rewritten quickly
Disadvantages: blocks cannot be modified
Read out the script data, use BeautifulSoup's xml parsing function to parse and modify
Note: Our Jmx script is actually a standard format xml
Advantages: Quickly find elements and modify them
Disadvantages: Need to be familiar with the usage of BeautifulSoup
by Beautiful Soup
Beautiful Soup
Beautiful Soup is a Python library that can extract data from HTML or XML files. When we use BeautifulSoup to parse xml or html, we can get a BeautifulSoup object, and we can complete the structured data of the original data by manipulating this object. For specific usage, please refer to this document .
Implementation
soup.find The sum function of bs4 is mainly used self.soup.find_all. Structured or modified data such as loops.string = num.
It is worth noting that find_all supports regular matching, and even if there is no suitable filter, you can also define a method that only accepts one element parameter.
The modified script will "T{}L{}R{}-{}_{}.jmx".format(thread_num, loop_num, ramp_time, self.src_script, self.get_time()) be saved in the form of , and the specific package is as follows:
import time
import os
from bs4 import BeautifulSoup
class OpJmx:
def __init__(self, file_name):
self.src_script = self._split_filename(file_name)
with open(file_name, "r") as f:
data = f.read()
self.soup = BeautifulSoup(data, "xml")
@staticmethod
def _split_filename(filename):
"""
新生成的文件兼容传入相对路径及文件名称
:param filename:
:return:
"""
relative = filename.split("/")
return relative[len(relative)-1].split(".jmx")[0]
def _theard_num(self):
"""
:return: 线程数据对象
"""
return self.soup.find("stringProp", {"name": {"ThreadGroup.num_threads"}})
def _ramp_time(self):
"""
:return: 启动所有线程时间配置对象
"""
return self.soup.find("stringProp", {"name": {"ThreadGroup.ramp_time"}})
def _bean_shell(self):
"""
:return: bean_shell对象
"""
return self.soup.find("stringProp", {"name": {"BeanShellSampler.query"}})
def _paths(self):
"""
:return: 请求路径信息对象
"""
return self.soup.find_all("stringProp", {"name": {"HTTPSampler.path"}})
def _methods(self):
"""
:return: 请求方法对象
"""
return self.soup.find_all("stringProp", {"name": {"HTTPSampler.method"}})
def _argument(self):
"""
:return: post请求参数对象
"""
# Argument.value 不唯一 通过HTTPArgument.always_encode找到
return self.soup.find_all("boolProp", {"name": {"HTTPArgument.always_encode"}})[0].find_next()
def _loops(self):
"""
循环次数，兼容forever 与具体次数
:return: 循环次数对象
"""
_loops = self.soup.find("stringProp", {"name": {"LoopController.loops"}})
if _loops:
pass
else:
_loops = self.soup.find("intProp", {"name": {"LoopController.loops"}})
return _loops
@staticmethod
def get_time():
return time.strftime("%Y-%m-%d@%X", time.localtime())
def get_bean_shell(self):
_str = self._bean_shell().string
logger.info("bean_shell: " + _str)
return _str
def set_bean_shell(self, new_bean_shell):
old_bean_shell = self._bean_shell()
old_bean_shell.string = new_bean_shell
def get_ramp_time(self):
_str = self._ramp_time().string
logger.info("ramp_time: " + _str)
return _str
@check_num
def set_ramp_time(self, num):
loops = self._ramp_time()
loops.string = num
def get_loops(self):
_str = self._loops().string
logger.info("loops: " + _str)
return _str
@check_num
def set_loops(self, num):
"""
:param num: -1 为一直循环，其他为具体循环次数
:return:
"""
loops = self._loops()
loops.string = num
def get_argument(self):
_str = self._argument().string
logger.info("argument: " + _str)
return _str
def set_argument(self, **kwargs):
"""
设置请求参数（JSON，传入字典）
:param kwargs:
:return:
"""
param = self._argument()
param.string = str(kwargs)
def get_thread_num(self):
_str = self._theard_num().string
logger.info("thread_num: " + _str)
return _str
@check_num
def set_thread_num(self, num):
"""
设置线程数信息
:param num:
:return:
"""
thread_num = self._theard_num()
thread_num.string = num
# print(self.soup.find_all("stringProp", {"name": {"ThreadGroup.num_threads"}})[0].string)
def mod_header(self, key, value, index=0):
"""
修改指定header的信息，默认修改第一个值
:param key:
:param value:
:param index:
:return:
"""
headers = self.soup.find_all("elementProp", {"elementType": {"Header"}})
headers[index].find("stringProp", {"name": {"Header.name"}}).string = key
headers[index].find("stringProp", {"name": {"Header.value"}}).string = value
# for header in headers:
# header.find("stringProp", {"name": {"Header.name"}}).string = key
# header.find("stringProp", {"name": {"Header.value"}}).string = value
def save_jmx(self):
logger.info("参数设置完毕，开始保存数据")
cur_path = os.path.dirname(os.path.realpath(__file__))
thread_num = self.get_thread_num()
loop_num = self.get_loops()
ramp_time = self.get_ramp_time()
script_name = "T{}L{}R{}-{}_{}.jmx".format(thread_num, loop_num, ramp_time, self.src_script, self.get_time())
script_path = os.path.join(cur_path, '..', 'script')
if not os.path.exists(script_path):
os.mkdir(script_path)
script_location = os.path.join(script_path, script_name)
logger.info("测试脚本已保存于 {}".format(script_location))
with open(script_location, "w") as f:
f.write(str(self.soup))
return script_name
if __name__ == '__main__':
jmx = OpJmx("templates/template.jmx")
argvs = sys.argv
len_argvs = len(argvs) - 1
if len_argvs == 0:
pass
elif len_argvs == 1:
jmx.set_thread_num(argvs[1])
elif len_argvs == 2:
jmx.set_thread_num(argvs[1])
jmx.set_loops(argvs[2])
elif len_argvs == 3:
jmx.set_thread_num(argvs[1])
jmx.set_loops(argvs[2])
jmx.set_ramp_time(argvs[3])
jmx.save_jmx()
To be continued...
Use string.Template character replacement
If it’s just a simple string replacement, use format or %s can also be done, the reason for choosing to use string.Template is that string.Template can automate matching rules, and can modify operators,
regardless of whether it is fstringor formatis used {}to locate keywords , {}has a specific meaning in jmx script itself.
Ideas:
- Modify key data in jmx script, using specific operators
- Define the relevant dictionary and
safe_substituteassign it using
Implementation
#! /usr/bin/python
# coding:utf-8
"""
@author:Bingo.he
@file: str_temp.py
@time: 2019/08/20
"""
import string
# with open("template_str.jmx", "r") as f:
# data = f.read()
set_value = {
"num_threads": 10,
"loops": 1011,
"ramp_time": 10
}
str_temp = """
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">%loops</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">%num_threads</stringProp>
<stringProp name="ThreadGroup.ramp_time">%ramp_time</stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
</ThreadGroup>
"""
class MyTemplate(string.Template):
# 修改操作符为"%"
delimiter = '%'
# 修改匹配规则(正则)
# idpattern = '[a-z]+_[a-z]+'
t = MyTemplate(str_temp)
print(t.safe_substitute(set_value))
output:
...
<stringProp name="LoopController.loops">1011</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">101</stringProp>
<stringProp name="ThreadGroup.ramp_time">10</stringProp>
...
use re.sub
str_temp = """
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">$loops</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">$num_threads</stringProp>
<stringProp name="ThreadGroup.ramp_time">$ramp_time</stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
</ThreadGroup>
"""
str_l = re.sub(r"\$loops", "101", str_temp)
str_t = re.sub(r"\$num_threads", "102", str_l)
str_r = re.sub(r"\$ramp_time", "103", str_t)
print(str_r)
output:
···
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">101</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">102</stringProp>
<stringProp name="ThreadGroup.ramp_time">103</stringProp>
···
extend
I believe everyone has noticed that we need to call it once every time we replace a parameter re.sub, and we need to use the output of the previous call as the next input, which is very similar to a recursive call. But today we do not introduce the method of recursive rewriting, but the way of using closures. The specific examples are as follows:
import re
def multiple_replace(text, adict):
rx = re.compile('|'.join(map(re.escape, adict)))
def one_xlat(match):
return adict[match.group(0)]
return rx.sub(one_xlat, text) # 每遇到一次匹配就会调用回调函数
# 把key做成了 |分割的内容，也就是正则表达式的OR
map1 = {'1': '2', '3': '4', '5': '6'}
_str = '113355'
print(multiple_replace(_str, map1))
There may be incorrect descriptions in the article, welcome to correct and supplement!
Data acquisition method
【Message 777】
Friends who want to get source code and other tutorial materials, please like + comment + bookmark , triple!
After three times in a row , I will send you private messages one by one in the comment area~