Python tools (guide package problem, pickle protocol problem, argparse usage, configparser usage, assert, difference difference set)

Python tools (guide package problem, pickle protocol problem, argparse usage, configparser usage, assert, difference difference set)

Python package import problem:

Relative path (prone to error):

# .同级目录 ..上级目录
from .resource import UserAgents
from .resource import PROXIES```

Absolute path:

from model.resource import UserAgents
from model.resource import PROXIES

pickle protocol problem

Python3.7 does not support pickle protocol 5:

ValueError: unsupported pickle protocol: 5

If the pkl file is generated in the environment of python3.8 (protocol 5), how to open the pkl file of protocol 5 in the environment of 3.7?

import pickle5 as pkl
pkl.load(文件名)

argparse usage (used to set running parameters)

import argparse

parser = argparse.ArgumentParser(description='dnn conf')
    parser.add_argument(
        '--conf_dir',           # “--”表示可选参数
        dest='conf_dir',
        default='conf/settings/',
        help='dnn config file dir')
        
args = parser.parse_args()

# vars()将传入的object转为字典类型
return vars(args)

para = args['conf_dir']

configparser usage (used to read the configuration file conf)

conf file structure:

[info]         # section info
data =2022-03-05  # option
purpose ="mmoe conf"
type = train

[tfrecord]    # section tfrecord
value_pairs = 
emb = 

[features]
emb = ex_zs:70:512:ex_zs:ex#kexia_bz:50000:512:kexia_bz:yian#she_bz:500:512:she_bz:yian#tai_bz:400:512:tai_bz:yian#mai_bz:400:512:mai_bz:yian#ex_age_gender78:60:32:ex_age_gender78:ex
value_pairs = kexia_bz:ex_zs:zxd|tfidf:avg#she_bz:ex_zs:gx|tfidf|avg
sim_emb =

usage:

conf_parser = configparser.ConfigParser()
conf_parser.read(conf_dir + conf_file)		# 读取文件

for section in self.conf_parser.sections():  # 读取所有Section
	self.conf_sections[section] = self.get_section_conf(section)

Note: All of the above are in the form of strings after being read in, and generally need to be converted:

def reset(self, section, option, f, default):
	try:
		self.conf_sections[section][option] = f(self.conf_sections[section][option])
		if not self.conf_sections[section][option]:
			self.conf_sections[section][option] = default
	except Exception as e:
		print("Reset conf_sections[%s][%s] failed, will use default value. Exception: %s" % (section, option, e))
		if section not in self.conf_sections:
			self.conf_sections[section] = {
    
    }
		self.conf_sections[section][option] = default
	finally:
		pass

assert usage (whether to let the program execute normally)

assert flag in ['train', 'predict']

If the flag is not in this list, an error is reported and the program stops.

difference set difference

type(target_list)=='list'
type(target_table_set)=='list'

rec_neg = list(target_table_set.differenct(set(target_list)))
rec_pos = target_list