内容简介：
本项目是对论文Simple Baselines for Human Pose Estimation and Tracking的复现，相比于hourglass，cpn等，该模型的网络结构看起来很直观简洁。网络结构就是在ResNet后加上几层Deconvolution直接生成热力图。相比于其他模型，就是使用Deconvolution替换了上采样结构。该模型的网络结构如下图所示：
博客推荐：Simple Baselines for Human Pose Estimation
文件结构
本项目的目录结构及文件功能如下：
root # 根目录
|--data # 存放数据
|--|--data9663 # 本项目挂载的数据集目录
|--|--coco # 解压后的coco数据集
|--|--|--annotations
|--|--|--images
|--lib # 包含reader和net部分
|--|--base_reader.py # 定义了数据部分公用的一些方法
|--|--coco_reader.py # 定义了coco数据集的reader
|--|--mpii_reader.py # 定义了mpii数据集的reader
|--|--pose_resnet.py # 定义了simple baseline的网络结构，
# 目前主干网络支持resnet50、resnet101、resnet152
|--utils # 包含evaluator和工具类部分
|--|--base_evaluator.py # 定义了evaluator的基类
|--|--mpii_evaluator.py # 针对mpii数据集的evaluator
|--|--transforms.py # 图像增强中使用的一些图像变换方法
|--|--nms_utils.py # coco_evaluator需要用到的方法
|--|--utility.py # 工具类
|--|--evaluator_builder.py # 根据不同的数据集构建evaluator并返回
|--|--coco_evaluator.py # 针对COCO数据集的evaluator
|--pretrained # 保存了主干网络预训练的模型
|--output # 保存了训练阶段输出的模型
|--test # 用于存放需要检测关键点的图像
|--results # 用于保存test.py输出的结果
|--freeze_model # 用于保存固化后的模型
|--infer_results #用于保存infer.py输出的结果
|--train.py # 训练
|--val.py # 验证
|--test.py # 对test目录下存放的图像进行关键点检测
|--freeze.py # 固化模型
|--infer.py # 使用固化后的模型对test目录下存放的图像进行关键点检测
如果需要训练自己的数据集可以参照coco数据集格式，将自己的数据集转化为coco数据集的格式，然后使用coco_reader和coco_evaluator
本示例由于展示需要，仅从coco数据集中抽取出了100张图像进行训练和测试，并且训练只进行1个epoch，因此模型的效果并不好。如果需要复现论文中的效果，可以在paddle_model_zoo中下载预训练过的模型，链接如下：paddle 源码地址
COCO val2017结果（使用的检测器在COCO val2017数据集上AP为56.4）
# 解压数据集
!unzip -qo data/data9663/coco.zip -d data/
print '数据解压完毕'
# 安装COCOAPI
!git clone https://github.com/cocodataset/cocoapi.git COCOAPI
!cd COCOAPI/PythonAPI && make install
!python COCOAPI/PythonAPI/setup.py install
print 'API安装完成'
# 下载预训练好的resnet50模型,并解压到pretrained
!wget http://paddle-imagenet-models.bj.bcebos.com/resnet_50_model.tar
!mkdir pretrained
!tar xf resnet_50_model.tar -C pretrained
!rm resnet_50_model.tar
print '预训练模型下载完成'
# 安装依赖的库
!pip install pathlib
print '准备工作完成'
数据解压完毕
fatal: destination path ‘COCOAPI’ already exists and is not an empty directory.
install pycocotools to the Python site-packages
python setup.py build_ext install
running build_ext
skipping ‘pycocotools/_mask.c’ Cython extension (up-to-date)
building ‘pycocotools._mask’ extension
creating build
creating build/common
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/pycocotools
gcc -pthread -B /opt/conda/envs/python27-paddle120-env/compiler_compat -Wl,–sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages/numpy/core/include -I…/common -I/opt/conda/envs/python27-paddle120-env/include/python2.7 -c …/common/maskApi.c -o build/temp.linux-x86_64-2.7/…/common/maskApi.o -Wno-cpp -Wno-unused-function -std=c99
…/common/maskApi.c: In function ‘rleToBbox’:
…/common/maskApi.c:141:31: warning: ‘xp’ may be used uninitialized in this function [-Wmaybe-uninitialized]
if(j%2==0) xp=x; else if(xp<x) { ys=0; ye=h-1; }
^
gcc -pthread -B /opt/conda/envs/python27-paddle120-env/compiler_compat -Wl,–sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages/numpy/core/include -I…/common -I/opt/conda/envs/python27-paddle120-env/include/python2.7 -c pycocotools/_mask.c -o build/temp.linux-x86_64-2.7/pycocotools/_mask.o -Wno-cpp -Wno-unused-function -std=c99
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/pycocotools
gcc -pthread -shared -B /opt/conda/envs/python27-paddle120-env/compiler_compat -L/opt/conda/envs/python27-paddle120-env/lib -Wl,-rpath=/opt/conda/envs/python27-paddle120-env/lib -Wl,–no-as-needed -Wl,–sysroot=/ build/temp.linux-x86_64-2.7/…/common/maskApi.o build/temp.linux-x86_64-2.7/pycocotools/_mask.o -L/opt/conda/envs/python27-paddle120-env/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/pycocotools/_mask.so
running install
running bdist_egg
running egg_info
writing requirements to pycocotools.egg-info/requires.txt
writing pycocotools.egg-info/PKG-INFO
writing top-level names to pycocotools.egg-info/top_level.txt
writing dependency_links to pycocotools.egg-info/dependency_links.txt
reading manifest file ‘pycocotools.egg-info/SOURCES.txt’
writing manifest file ‘pycocotools.egg-info/SOURCES.txt’
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying pycocotools/mask.py -> build/lib.linux-x86_64-2.7/pycocotools
copying pycocotools/cocoeval.py -> build/lib.linux-x86_64-2.7/pycocotools
copying pycocotools/init.py -> build/lib.linux-x86_64-2.7/pycocotools
copying pycocotools/coco.py -> build/lib.linux-x86_64-2.7/pycocotools
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/pycocotools
copying build/lib.linux-x86_64-2.7/pycocotools/mask.py -> build/bdist.linux-x86_64/egg/pycocotools
copying build/lib.linux-x86_64-2.7/pycocotools/_mask.so -> build/bdist.linux-x86_64/egg/pycocotools
copying build/lib.linux-x86_64-2.7/pycocotools/cocoeval.py -> build/bdist.linux-x86_64/egg/pycocotools
copying build/lib.linux-x86_64-2.7/pycocotools/init.py -> build/bdist.linux-x86_64/egg/pycocotools
copying build/lib.linux-x86_64-2.7/pycocotools/coco.py -> build/bdist.linux-x86_64/egg/pycocotools
byte-compiling build/bdist.linux-x86_64/egg/pycocotools/mask.py to mask.pyc
byte-compiling build/bdist.linux-x86_64/egg/pycocotools/cocoeval.py to cocoeval.pyc
byte-compiling build/bdist.linux-x86_64/egg/pycocotools/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/pycocotools/coco.py to coco.pyc
creating stub loader for pycocotools/_mask.so
byte-compiling build/bdist.linux-x86_64/egg/pycocotools/_mask.py to _mask.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying pycocotools.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pycocotools.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pycocotools.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pycocotools.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pycocotools.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents…
creating ‘dist/pycocotools-2.0-py2.7-linux-x86_64.egg’ and adding ‘build/bdist.linux-x86_64/egg’ to it
removing ‘build/bdist.linux-x86_64/egg’ (and everything under it)
Processing pycocotools-2.0-py2.7-linux-x86_64.egg
Copying pycocotools-2.0-py2.7-linux-x86_64.egg to /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Adding pycocotools 2.0 to easy-install.pth file
Installed /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages/pycocotools-2.0-py2.7-linux-x86_64.egg
Processing dependencies for pycocotools2.0
Searching for matplotlib2.2.3
Best match: matplotlib 2.2.3
Adding matplotlib 2.2.3 to easy-install.pth file
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for Cython==0.29.6
Best match: Cython 0.29.6
Adding Cython 0.29.6 to easy-install.pth file
Installing cython script to /opt/conda/envs/python27-paddle120-env/bin
Installing cygdb script to /opt/conda/envs/python27-paddle120-env/bin
Installing cythonize script to /opt/conda/envs/python27-paddle120-env/bin
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for setuptools==41.0.0
Best match: setuptools 41.0.0
Adding setuptools 41.0.0 to easy-install.pth file
Installing easy_install script to /opt/conda/envs/python27-paddle120-env/bin
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for kiwisolver==1.0.1
Best match: kiwisolver 1.0.1
Adding kiwisolver 1.0.1 to easy-install.pth file
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for numpy==1.16.2
Best match: numpy 1.16.2
Adding numpy 1.16.2 to easy-install.pth file
Installing f2py2 script to /opt/conda/envs/python27-paddle120-env/bin
Installing f2py script to /opt/conda/envs/python27-paddle120-env/bin
Installing f2py2.7 script to /opt/conda/envs/python27-paddle120-env/bin
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for pyparsing==2.4.0
Best match: pyparsing 2.4.0
Adding pyparsing 2.4.0 to easy-install.pth file
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for pytz==2018.9
Best match: pytz 2018.9
Adding pytz 2018.9 to easy-install.pth file
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for backports.functools-lru-cache==1.5
Best match: backports.functools-lru-cache 1.5
Adding backports.functools-lru-cache 1.5 to easy-install.pth file
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for six==1.12.0
Best match: six 1.12.0
Adding six 1.12.0 to easy-install.pth file
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for cycler==0.10.0
Best match: cycler 0.10.0
Adding cycler 0.10.0 to easy-install.pth file
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for subprocess32==3.5.3
Best match: subprocess32 3.5.3
Adding subprocess32 3.5.3 to easy-install.pth file
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Searching for python-dateutil==2.8.0
Best match: python-dateutil 2.8.0
Adding python-dateutil 2.8.0 to easy-install.pth file
Using /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages
Finished processing dependencies for pycocotools==2.0
rm -rf build
running install
running bdist_egg
running egg_info
writing requirements to pycocotools.egg-info/requires.txt
writing pycocotools.egg-info/PKG-INFO
writing top-level names to pycocotools.egg-info/top_level.txt
writing dependency_links to pycocotools.egg-info/dependency_links.txt
error: package directory ‘pycocotools’ does not exist
API安装完成
–2019-09-06 11:36:31-- http://paddle-imagenet-models.bj.bcebos.com/resnet_50_model.tar
Resolving paddle-imagenet-models.bj.bcebos.com (paddle-imagenet-models.bj.bcebos.com)… 182.61.200.229, 182.61.200.195
Connecting to paddle-imagenet-models.bj.bcebos.com (paddle-imagenet-models.bj.bcebos.com)|182.61.200.229|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 102717440 (98M) [application/x-tar]
Saving to: ‘resnet_50_model.tar’
resnet_50_model.tar 100%[===================>] 97.96M 35.1MB/s in 2.8s
2019-09-06 11:36:35 (35.1 MB/s) - ‘resnet_50_model.tar’ saved [102717440/102717440]
mkdir: cannot create directory ‘pretrained’: File exists
预训练模型下载完成
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won’t be maintained after that date. A future version of pip will drop support for Python 2.7.
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Collecting pathlib
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by ‘ReadTimeoutError(“HTTPSConnectionPool(host=‘mirrors.ustc.edu.cn’, port=443): Read timed out. (read timeout=15)”,)’: /pypi/web/simple/pathlib/
Downloading https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ac/aa/9b065a76b9af472437a0059f77e8f962fe350438b927cb80184c32f075eb/pathlib-1.0.1.tar.gz (49kB)
100% |████████████████████████████████| 51kB 7.0MB/s ta 0:00:011
Building wheels for collected packages: pathlib
Building wheel for pathlib (setup.py) … done
Stored in directory: /home/aistudio/.cache/pip/wheels/47/98/36/d70c66120fd305cede653371060e18ae398f89ea7452b13923
Successfully built pathlib
Installing collected packages: pathlib
Successfully installed pathlib-1.0.1
准备工作完成
# 在pretrained目录下保存的预训练模型的基础上训练simple baseline网络
# 注意训练中使用的resnet需要和预训练模型保持一致，例如：训练中使用resnet50作为骨干网络，
# pretrained中也需要是resenet50的预训练模型
# 具体的训练参数可以在train中查看和修改
!python train.py --dataset 'coco' --num_epochs 1
----------- Configuration Arguments -----------
batch_size: 12
checkpoint: None
dataset: coco
kp_dim: 17
lr: 0.001
lr_strategy: piecewise_decay
model_save_dir: output
num_epochs: 1
pretrained_model: pretrained/resnet_50/115
total_images: 100
use_gpu: True
with_mem_opt: True
=> LR will be dropped at the epoch of [90, 120]
memory_optimize is deprecated. Use CompiledProgram and Executor
W0906 11:41:52.145272 249 device_context.cc:259] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 9.2, Runtime API Version: 9.0
W0906 11:41:52.150189 249 device_context.cc:267] device: 0, cuDNN Version: 7.3.
I0906 11:41:52.379009 249 parallel_executor.cc:329] The number of CUDAPlace, which is used in ParallelExecutor, is 1. And the Program will be copied 1 copies
I0906 11:41:52.441627 249 build_strategy.cc:340] SeqOnlyAllReduceOps:0, num_trainers:1
loading annotations into memory…
Done (t=0.03s)
creating index…
index created!
=> classes: [‘background’, u’person’]
=> num_images: 500
generating coco gt_db…
=> num db: 430
=> num selected db: 416
Epoch [ 0/ 0] LR: 0.0000000000 Loss = 0.00155
Epoch [ 1/ 0] LR: 0.0010000000 Loss = 0.00568
Epoch [ 2/ 0] LR: 0.0010000000 Loss = 0.00140
Epoch [ 3/ 0] LR: 0.0010000000 Loss = 0.00173
Epoch [ 4/ 0] LR: 0.0010000000 Loss = 0.00203
Epoch [ 5/ 0] LR: 0.0010000000 Loss = 0.00233
Epoch [ 6/ 0] LR: 0.0010000000 Loss = 0.00222
Epoch [ 7/ 0] LR: 0.0010000000 Loss = 0.00173
Epoch [ 8/ 0] LR: 0.0010000000 Loss = 0.00155
Epoch [ 9/ 0] LR: 0.0010000000 Loss = 0.00091
Epoch [ 10/ 0] LR: 0.0010000000 Loss = 0.00164
Epoch [ 11/ 0] LR: 0.0010000000 Loss = 0.00137
Epoch [ 12/ 0] LR: 0.0010000000 Loss = 0.00155
Epoch [ 13/ 0] LR: 0.0010000000 Loss = 0.00144
Epoch [ 14/ 0] LR: 0.0010000000 Loss = 0.00141
Epoch [ 15/ 0] LR: 0.0010000000 Loss = 0.00144
Epoch [ 16/ 0] LR: 0.0010000000 Loss = 0.00127
Epoch [ 17/ 0] LR: 0.0010000000 Loss = 0.00147
Epoch [ 18/ 0] LR: 0.0010000000 Loss = 0.00124
Epoch [ 19/ 0] LR: 0.0010000000 Loss = 0.00131
Epoch [ 20/ 0] LR: 0.0010000000 Loss = 0.00130
Epoch [ 21/ 0] LR: 0.0010000000 Loss = 0.00155
Epoch [ 22/ 0] LR: 0.0010000000 Loss = 0.00130
Epoch [ 23/ 0] LR: 0.0010000000 Loss = 0.00130
Epoch [ 24/ 0] LR: 0.0010000000 Loss = 0.00139
Epoch [ 25/ 0] LR: 0.0010000000 Loss = 0.00130
Epoch [ 26/ 0] LR: 0.0010000000 Loss = 0.00137
Epoch [ 27/ 0] LR: 0.0010000000 Loss = 0.00104
Epoch [ 28/ 0] LR: 0.0010000000 Loss = 0.00146
Epoch [ 29/ 0] LR: 0.0010000000 Loss = 0.00129
Epoch [ 30/ 0] LR: 0.0010000000 Loss = 0.00130
Epoch [ 31/ 0] LR: 0.0010000000 Loss = 0.00115
Epoch [ 32/ 0] LR: 0.0010000000 Loss = 0.00146
Epoch [ 33/ 0] LR: 0.0010000000 Loss = 0.00118
Epoch [ 34/ 0] LR: 0.0010000000 Loss = 0.00132
# 使用output中保存的训练阶段暂存的模型进行测试
# 待检测的图像需要放在test目录下
# 检测后的含有关键点的图像放在result目录下
# 具体的参数可以在test.py中查看
# 注意如果test.py中的flip_test参数设置为False，将不会输出检测结果图像
!python test.py --checkpoint 'output/simplebase-coco/10'
----------- Configuration Arguments -----------
batch_size: 32
checkpoint: output/simplebase-coco/10
dataset: coco
flip_test: True
kp_dim: 17
shift_heatmap: False
use_gpu: True
with_mem_opt: True
memory_optimize is deprecated. Use CompiledProgram and Executor
W0906 11:42:54.990016 321 device_context.cc:259] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 9.2, Runtime API Version: 9.0
W0906 11:42:54.994679 321 device_context.cc:267] device: 0, cuDNN Version: 7.3.
WARNING:root:
You can try our memory optimize feature to save your memory usage:
# create a build_strategy variable to set memory optimize option
build_strategy = compiler.BuildStrategy()
build_strategy.enable_inplace = True
build_strategy.memory_optimize = True
pass the build_strategy to with_data_parallel API
compiled_prog = compiler.CompiledProgram(main).with_data_parallel(
loss_name=loss.name, build_strategy=build_strategy)
!!! Memory optimize is our experimental feature !!!
some variables may be removed/reused internal to save memory usage,
in order to fetch the right value of the fetch_list, please set the
persistable property to true for each variable in fetch_list
Sample
conv1 = fluid.layers.conv2d(data, 4, 5, 1, act=None)
# if you need to fetch conv1, then:
conv1.persistable = True
I0906 11:42:55.210430 321 parallel_executor.cc:329] The number of CUDAPlace, which is used in ParallelExecutor, is 1. And the Program will be copied 1 copies
I0906 11:42:55.219099 321 build_strategy.cc:340] SeqOnlyAllReduceOps:0, num_trainers:1
Processing batch #0
The result has been output to ‘/results’
# 将训练过程中暂存的模型固化，具体需要固化的模型路径在freeze.py中的path参数指定
!python freeze.py --pretrained_model 'output/simplebase-coco/10'
# 使用freeze_model目录中保存的固化后的模型进行测试，并将检测结果输出到infer_results目录下
!python infer.py
Processing sample #0
Processing sample #1
Processing sample #2
The result has been output to ‘/infer_results’
