[Code Notes] Summary of the basic usage of Python's argparse module

Summary of the basic usage of Python's argparse module

1 Introduction

argparse is a library for processing command-line arguments in the Python standard.

2. use

2.1 Initialize the parameter parser object

The ArgumentParser object saves all the necessary information to parse the command line parameters into the corresponding python data types, where the description parameter means to display help information on the command line. This object also has other parameters, which are generally seldom used and will not be described here. For details, see the official documentation .

import argparse

# 创建参数解析器：一个用于把命令行字符串解析成Python对象的对象
parser = argparse.ArgumentParser(description="这是一个argparser使用示例程序")

2.2 Add parameters

Use the add_argument() method to add parameters. The parameters of this method are described below.

Add_argument() method parameter description

• name or flags - The required parameter name or optional parameter identifier, which must be the first parameter of the add_argument() method.

  Command line parameters are divided into positional parameters and option parameters:

  • Positional (required) parameter : - or – is not required in front, this method requires the input of the parameter, inputting the parameter does not need to specify the parameter name, if it is specified, an error will be reported, and the interpreter will automatically assign the input parameter to the corresponding parameter;

  • Optional parameters : short parameters specified by a -, such as -e, to simplify parameter input; long parameters specified by –, such as –epoch. Entering short parameters and long parameters has the same effect when inputting on the command line . It should be noted that if you want to retrieve the value through the parsed parameters, you must use the name with – .

• dest - the variable name corresponding to the parameter in the program

• action - Indicates the way the value is assigned to the key. The bool type is used here. Action means the behavior when the specified parameter appears in the read parameters .

  • store : The default action mode, store the value to the specified variable.
  • store_const : The stored value is specified in the const part of the parameter, which is mostly used to implement non-Boolean command line flags.
  • store_true / store_false : Boolean switch. store_true is False by default and true for input. store_false instead. Usage: When a parameter with the action attribute appears in the command line (terminal), the value of the parameter is ture represented by "store_true"; when there is no parameter with the action attribute in the command line, the value of the parameter is "store_true" "The relative value represented is false.
  • append : store values ​​into a list, this parameter can be reused.
  • append_const : Store the value to the list, and the stored value is specified in the const part of the parameter.
  • count : The number of abbreviated inputs for statistical parameters.
  • version : Print version information and exit.

• help - description information of the parameter

• required - indicates whether the parameter must be input, only optional , and the default is False .

• type - specifies the command line parameter data type

• choices - Indicates the value range of command line parameters, and its value is generally a list. The type of elements in the choices list should be compatible with the type specified by type.

• nargs - Specify how many values ​​are behind this parameter, the default is 1.

  • If it is set to a number, such as 2, it means accepting 2 values;
  • If set to '?' then only 1 or 0 values ​​are accepted;
  • If it is set to '*', it can be followed by 0 or more values;
  • If it is set to '+', it means that one or more values ​​can be connected;

• default - Default values ​​for required and optional parameters.

Commonly used parameters are mainly name or flags, action, help, type, default.

Example 1

import argparse

# 创建参数解析器：一个用于把命令行字符串解析成Python对象的对象
parser = argparse.ArgumentParser(description="这是一个argparser使用示例程序")
# 添加一个可选参数，使用时，输入-lr和--learning_rate效果一样，但是解析时必须用learning_rate取出
parser.add_argument('-lr', '--learning_rate', default=1e-5, type=float, help='学习率')
# 添加一个可选参数，输入必须是bert/roberta/albert中的一个，否则报错
parser.add_argument('--model', default='bert', choices=['bert', 'roberta', 'albert'], help='模型')
# 添加一个可选参数，没有其他约束
parser.add_argument('--n_heads', type=int, help='注意力头数')
# 添加两个位置参数，必须输入，会按顺序赋值。输入该参数不需要指定参数名称，指定反而报错
parser.add_argument('epoch')
parser.add_argument('n_layers')

2.3 Parsing parameters

parse_args()

The added arguments are parsed using parse_args(), which examines the command line, converts each argument to the appropriate type and invokes the appropriate action. In most cases this means that a simple Namespace object will be constructed from the attributes parsed from the command line.

Example 2

Follow the example 1 code

args = parser.parse_args()
print(args)
print(args.learning_rate)
print(args.model)
print(args.epoch)
print(args.n_layers)

Enter at the command line :

python main.py -lr 2e-5 --model albert 30 6

This command means: assign 2e-5 to learning_rate, assign albert to model, automatically assign 30 to epoch, and automatically assign 6 to n_layers.

The output is as follows :

Namespace(learning_rate=2e-05, model='albert', n_heads=None, epoch='30', n_layers='6')
2e-05
albert
30
6

• In addition, the program and parameter description can be viewed through the '-h' command, and the program will not be executed.

Enter at the command line :

 python main.py -h
 # 或python main.py --help

The output is as follows :

usage: main.py [-h] [-lr LEARNING_RATE] [--model {
    
    bert,roberta,albert}] [--n_heads N_HEADS] epoch n_layers

这是一个argparser使用示例程序

positional arguments:
  epoch
  n_layers

optional arguments:
  -h, --help            show this help message and exit
  -lr LEARNING_RATE, --learning_rate LEARNING_RATE
                        学习率
  --model {
    
    bert,roberta,albert}
                        模型
  --n_heads N_HEADS     注意力头数

Example 3: Description of action parameters

parser = argparse.ArgumentParser(description="这是一个argparser使用示例程序")
# 添加一个布尔参数，名为train，当命令中出现这个参数时，即为false，否则为True。
parser.add_argument('--train', action='store_false',default=True)
args = parser.parse_args()
print(args.train)

Enter the command and output as follows:

python main.py
## 输出：True
python main.py --train
## 输出：False

It can be seen from the above example that when default and action_false are satisfied at the same time, the priority of action is higher than default . That is, the '-train' command appears, and the assignment is False instead of the default True.

parse_known_args()

This method can also be used to parse parameters, the method returns two parameters:

args, unparsed = parser.parse_known_args()

args is the namespace space, including the command line parameters defined by the program, and unparsed is the command line parameters not defined by the program . The difference from the above function is that if the command line input has no defined parameters , no error will be reported (but parser_args() will report an error), and a list will be returned to unparsed, which contains undefined input parameters.

Application Scenario

Your parameter setting may not be done within a function or package call .

In many cases, we may need to determine the settings of other parameters based on some input parameters . For example, in deep learning, we may determine which model code we call based on the incoming model setting – model, and then there will be some parameter settings in the model.

Then there will be a situation at this time, that is, all the parameter values ​​that need to be set will be passed in to the running command, but sometimes it may be necessary to perform some operations when only the basic parameter settings are obtained before continuing to import the remaining parameter settings.

The function of the parse_known_args() method is that when only the basic settings are obtained, if other configurations that will be used later are passed in the running command, no error will be reported; instead, the extra parts will be saved for later use .

Example 4

import argparse

parser = argparse.ArgumentParser(description="这是一个argparser使用示例程序")
parser.add_argument('--model', default='bert')
parser.add_argument('batch_size')

args, unparsed = parser.parse_known_args()

print(args)
print(unparsed)

Enter the command :

python main.py --model roberta 32 --epoch 100 --lr 1e-5

The output is as follows :

Namespace(model='roberta', batch_size='32')
['--epoch', '100', '--lr', '1e-5']

It can be seen that the parameters epoch and lr that are not added to the parser are saved in the form of a list (list).