【Object Detection】【maskrcnn_benckmark】配置文件

1. 配置文件的工作原理

maskrcnn_benchmark 使用配置文件设置模型、数据、优化器等选项。.../maskrcnn_benchmark/maskrcnn_benchmark/config/ 下的 defaults.py 定义并设置了默认选项。
在 defaults.py 中，使用 yacs 中的 CfgNode 类的一个实例对象 _C 作为保存配置信息的容器.

# .../yacs/config.py

class CfgNode(dict):
    """
    CfgNode represents an internal node in the configuration tree. It's a simple
    dict-like container that allows for attribute-based access to keys.
    """
    ...

# .../maskrcnn_benchmark/maskrcnn_benchmark/config/defaults.py
# 第4行
from yacs.config import CfgNode as CN

...

# 第21行
# CfgNode 的实例 _C 保存了所有配置信息。
_C = CN()

在 ../maskrcnn_benchmark/maskrcnn_benchmark/config/__init__.py 中， 将保存配置信息的变量 _C 封装为 cfg ，cfg 本质上是 CfgNode 类的一个实例对象.

# .../maskrcnn_benchmark/maskrcnn_benchmark/config/__init__.py

from .defaults import _C as cfg

在训练或测试脚本中，导入 cfg，作为默认配置信息，使用 CfgNode 类的实例方法 merge_from_file() 从配置文件获取配置信息，merge_from_list() 从命令行接收配置信息.

# .../maskrcnn_benchmark/tools/train_net.py

#  14行
from maskrcnn_benchmark.config import cfg

...

    # 146行
    cfg.merge_from_file(args.config_file)
    cfg.merge_from_list(args.opts)

args.config_file 从命令行接收配置文件的路径，args.opts 从命令行接收一个保存配置文件信息的列表格式.
一般修改配置文件就行了，args.opts 较少使用 .
配置文件是一个 yaml 格式文件。
在 ../maskrcnn_benchmark/configs/ 保存了所有的论文实现模型的配置文件，使用对应的配置文件配合训练脚本，即可复现论文中的模型.
也可以自定义配置文件，在自己的数据集上训练对应的模型.

2. 配置文件的内容信息

我将 .../maskrcnn_benchmark/maskrcnn_benchmark/config/defaults.py 中的配置信息的内容转化为 defaults.yaml，并按 defaults.py 顺序重新排版如下:

MODEL:
  RPN_ONLY: false    # True for training RPN only, False for training whole model
  MASK_ON: false    # True for instance segmentation task enabled, False for not
  RETINANET_ON: false    # True for using Retinanet, False for not
  KEYPOINT_ON: false    # True for key point task enabled, False for not
  DEVICE: cuda    # cuda or cpu
  META_ARCHITECTURE: GeneralizedRCNN
  CLS_AGNOSTIC_BBOX_REG: false
  WEIGHT: ''
  BACKBONE:
    CONV_BODY: R-50-C4
    FREEZE_CONV_BODY_AT: 2
  FPN:
    USE_GN: false
    USE_RELU: false
  GROUP_NORM:
    DIM_PER_GP: -1
    NUM_GROUPS: 32
    EPSILON: 1.0e-05
  RPN:
    USE_FPN: false
    ANCHOR_SIZES: (32, 64, 128, 256, 512)
    ANCHOR_STRIDE: (16,)
    ASPECT_RATIOS: (0.5, 1.0, 2.0)
    STRADDLE_THRESH: 0
    FG_IOU_THRESHOLD: 0.7
    BG_IOU_THRESHOLD: 0.3
    BATCH_SIZE_PER_IMAGE: 256
    POSITIVE_FRACTION: 0.5
    PRE_NMS_TOP_N_TRAIN: 12000
    PRE_NMS_TOP_N_TEST: 6000
    POST_NMS_TOP_N_TRAIN: 2000
    POST_NMS_TOP_N_TEST: 1000
    NMS_THRESH: 0.7
    MIN_SIZE: 0
    FPN_POST_NMS_TOP_N_TRAIN: 2000
    FPN_POST_NMS_TOP_N_TEST: 2000
    FPN_POST_NMS_PER_BATCH: true
    RPN_HEAD: SingleConvRPNHead
  ROI_HEADS:
    USE_FPN: false
    FG_IOU_THRESHOLD: 0.5
    BG_IOU_THRESHOLD: 0.5
    BBOX_REG_WEIGHTS: (10.0, 10.0, 5.0, 5.0)
    BATCH_SIZE_PER_IMAGE: 512
    POSITIVE_FRACTION: 0.25
    SCORE_THRESH: 0.05
    NMS: 0.5
    DETECTIONS_PER_IMG: 100
  ROI_BOX_HEAD:
    FEATURE_EXTRACTOR: ResNet50Conv5ROIFeatureExtractor
    PREDICTOR: FastRCNNPredictor
    POOLER_RESOLUTION: 14
    POOLER_SAMPLING_RATIO: 0
    POOLER_SCALES: (1.0 / 16,)
    NUM_CLASSES: 81
    MLP_HEAD_DIM: 1024
    USE_GN: false
    DILATION: 1
    CONV_HEAD_DIM: 256
    NUM_STACKED_CONVS: 4
  ROI_MASK_HEAD:
    FEATURE_EXTRACTOR: ResNet50Conv5ROIFeatureExtractor
    PREDICTOR: MaskRCNNC4Predictor
    POOLER_RESOLUTION: 14
    POOLER_SAMPLING_RATIO: 0
    POOLER_SCALES: (1.0 / 16,)
    MLP_HEAD_DIM: 1024
    CONV_LAYERS: (256, 256, 256, 256)
    RESOLUTION: 14
    SHARE_BOX_FEATURE_EXTRACTOR: true
    POSTPROCESS_MASKS: false
    POSTPROCESS_MASKS_THRESHOLD: 0.5
    DILATION: 1
    USE_GN: false
  ROI_KEYPOINT_HEAD:
    FEATURE_EXTRACTOR: KeypointRCNNFeatureExtractor
    PREDICTOR: KeypointRCNNPredictor
    POOLER_RESOLUTION: 14
    POOLER_SAMPLING_RATIO: 0
    POOLER_SCALES: (1.0 / 16,)
    MLP_HEAD_DIM: 1024
    CONV_LAYERS: (512, 512, 512, 512, 512, 512, 512, 512)
    RESOLUTION: 14
    NUM_CLASSES: 17
    SHARE_BOX_FEATURE_EXTRACTOR: true
  RESNETS:
    NUM_GROUPS: 1
    WIDTH_PER_GROUP: 64
    STRIDE_IN_1X1: true
    TRANS_FUNC: BottleneckWithFixedBatchNorm
    STEM_FUNC: StemWithFixedBatchNorm
    RES5_DILATION: 1
    BACKBONE_OUT_CHANNELS: 1024
    RES2_OUT_CHANNELS: 256
    STEM_OUT_CHANNELS: 64
    STAGE_WITH_DCN: (false, false, false, false)
    WITH_MODULATED_DCN: false
    DEFORMABLE_GROUPS: 1
  RETINANET:
    NUM_CLASSES: 81
    ANCHOR_SIZES: (32, 64, 128, 256, 512)
    ASPECT_RATIOS: (0.5, 1.0, 2.0)
    ANCHOR_STRIDES: (8, 16, 32, 64, 128)
    STRADDLE_THRESH: 0
    OCTAVE: 2.0
    SCALES_PER_OCTAVE: 3
    USE_C5: true
    NUM_CONVS: 4
    BBOX_REG_WEIGHT: 4.0
    BBOX_REG_BETA: 0.11
    PRE_NMS_TOP_N: 1000
    FG_IOU_THRESHOLD: 0.5
    BG_IOU_THRESHOLD: 0.4
    LOSS_ALPHA: 0.25
    LOSS_GAMMA: 2.0
    PRIOR_PROB: 0.01
    INFERENCE_TH: 0.05
    NMS_TH: 0.4
  FBNET:
    ARCH: default
    ARCH_DEF: ''
    BN_TYPE: bn
    SCALE_FACTOR: 1.0
    WIDTH_DIVISOR: 1
    DW_CONV_SKIP_BN: true
    DW_CONV_SKIP_RELU: true
    DET_HEAD_LAST_SCALE: 1.0
    DET_HEAD_BLOCKS: []
    DET_HEAD_STRIDE: 0
    KPTS_HEAD_LAST_SCALE: 0.0
    KPTS_HEAD_BLOCKS: []
    KPTS_HEAD_STRIDE: 0
    MASK_HEAD_LAST_SCALE: 0.0
    MASK_HEAD_BLOCKS: []
    MASK_HEAD_STRIDE: 0
    RPN_HEAD_BLOCKS: 0
    RPN_BN_TYPE: ''
INPUT:
  MIN_SIZE_TRAIN: (800,)
  MAX_SIZE_TRAIN: 1333
  MIN_SIZE_TEST: 800
  MAX_SIZE_TEST: 1333
  PIXEL_MEAN: (102.9801, 115.9465, 122.7717)
  PIXEL_STD: (1.0, 1.0, 1.0)
  TO_BGR255: true
  BRIGHTNESS: 0.0
  CONTRAST: 0.0
  SATURATION: 0.0
  HUE: 0.0
  HORIZONTAL_FLIP_PROB_TRAIN: 0.5
  VERTICAL_FLIP_PROB_TRAIN: 0.0
DATASETS:
  TRAIN: []
  TEST: []
DATALOADER:
  NUM_WORKERS: 4
  SIZE_DIVISIBILITY: 0
  ASPECT_RATIO_GROUPING: true
SOLVER:
  MAX_ITER: 40000
  BASE_LR: 0.001
  BIAS_LR_FACTOR: 2
  MOMENTUM: 0.9
  WEIGHT_DECAY: 0.0005
  WEIGHT_DECAY_BIAS: 0
  GAMMA: 0.1
  STEPS: (30000,)
  WARMUP_FACTOR: 1.0 / 3
  WARMUP_ITERS: 500
  WARMUP_METHOD: linear
  CHECKPOINT_PERIOD: 2500
  TEST_PERIOD: 0
  IMS_PER_BATCH: 16
TEST:
  EXPECTED_RESULTS: []
  EXPECTED_RESULTS_SIGMA_TOL: 4
  IMS_PER_BATCH: 8
  DETECTIONS_PER_IMG: 100
  BBOX_AUG:
    ENABLED: false
    H_FLIP: false
    SCALES: []
    MAX_SIZE: 4000
    SCALE_H_FLIP: false
OUTPUT_DIR: .
PATHS_CATALOG: /home/lucifer/Projects/05_maskrcnn-benchmark/maskrcnn-benchmark/maskrcnn_benchmark/config/paths_catalog.py
DTYPE: float32
AMP_VERBOSE: false

3. cfg python 对象与 yaml 配置文件的相互转换

（1）cfg 到 yaml

from maskrcnn_benchmark.config import cfg

yaml_file_path = 'defaults.yaml'
with open(yaml_file_path, 'w') as yaml_file:
    cfg.dump(stream=yaml_file)

（2）yaml 到 cfg

from yacs.config import CfgNode as CN
# or
from maskrcnn_benchmark.config import cfg

yaml_file_path = 'e2e_faster_rcnn_self.yaml`'
with open(yaml_file_path, 'r') as yaml_file:
    cfg = CN.load_cfg(yaml_file)
    # or
    cfg = cfg.load_cfg(yaml_file)

load_cfg() 是类方法，不会改动 cfg.

4. 关于 yacs 和 yaml

使用配置文件，可以使训练模型更为便利和灵活。maskrcnn_benchmark 的配置文件基于 yacs 实现。 yacs 是一个轻量化的配置文件管理库，它基于 PyYAML 库实现。PyYaml 可以轻松的在 Python 与 YAML 之间交互信息.

扫描二维码关注公众号，回复： 9379439 查看本文章

4.1 PyYAML 的简单介绍

（1）yaml.safe_dump()

import yaml

data = {
    'name': 'Lucy',
    'age': 18,
    'birth': [2000, 12, 26],
    'mother': {
        'name': 'Kitty',
        'age': 30,
        'birth': [1988, 1, 26]
    }
}
yaml_file_path = 'data.yaml'
with open(yaml_file_path, 'w') as yaml_file:
    yaml.safe_dump(data, yaml_file)
    content = yaml.safe_dump(data)

print(content)

输出

age: 18
birth:
- 2000
- 12
- 26
mother:
  age: 30
  birth:
  - 1988
  - 1
  - 26
  name: Kitty
name: Lucy

得到 data.yaml

age: 18
birth:
- 2000
- 12
- 26
mother:
  age: 30
  birth:
  - 1988
  - 1
  - 26
  name: Kitty
name: Lucy

（2）yaml.save_load()

设置 data.yaml 为

age: 18
birth:
- 2000
- 12
- 26
mother:
  age: 30
  birth:
  - 1988
  - 1
  - 26
  name: Kitty
name: Lucy
father:
  age: 34
  birth: [1984, 12, 26]

代码

import yaml

yaml_file_path = 'data.yaml'
with open(yaml_file_path, 'r') as yaml_file:
    content = yaml.safe_load(yaml_file)

print(content)

输出

<class 'dict'>