caffe训练自己的数据
1数据准备
从网上下载了猫鸟数据，训练图片各40张，测试数据各10张。
接着准备训练与验证用的标注文件，由于图片数量较少，就用手工编写。
随后在caffe-master创建mytest文件夹，然后将imagenet的crea_imagenet.sh复制到该文件夹下，进行路径设置(不知为何，我这设置绝对路径才不会出错)，运行该sh。
#!/usr/bin/envsh
#Create the imagenet lmdb inputs
#N.B. set the path to the imagenet train + val data dirs
set-e
EXAMPLE=/home/caisheng/Downloads/caffe-master/examples/mytest
DATA=/home/caisheng/Downloads/caffe-master/data/my
TOOLS=/home/caisheng/Downloads/caffe-master/build/tools
TRAIN_DATA_ROOT=/home/caisheng/Downloads/caffe-master/data/my/train/
VAL_DATA_ROOT=/home/caisheng/Downloads/caffe-master/data/my/val/
#Set RESIZE=true to resize the images to 256x256. Leave as false ifimages have
#already been resized using another tool.
RESIZE=true
if$RESIZE; then
RESIZE_HEIGHT=256
RESIZE_WIDTH=256
else
RESIZE_HEIGHT=0
RESIZE_WIDTH=0
fi
if [! -d "$TRAIN_DATA_ROOT" ]; then
echo "Error: TRAIN_DATA_ROOT is not a path to a directory:$TRAIN_DATA_ROOT"
echo "Set the TRAIN_DATA_ROOT variable in create_imagenet.sh tothe path" \
"where the ImageNet training data is stored."
exit 1
fi
if [! -d "$VAL_DATA_ROOT" ]; then
echo "Error: VAL_DATA_ROOT is not a path to a directory:$VAL_DATA_ROOT"
echo "Set the VAL_DATA_ROOT variable in create_imagenet.sh tothe path" \
"where the ImageNet validation data is stored."
exit 1
fi
echo"Creating train lmdb..."
GLOG_logtostderr=1$TOOLS/convert_imageset \
--resize_height=$RESIZE_HEIGHT \
--resize_width=$RESIZE_WIDTH \
--shuffle \
$TRAIN_DATA_ROOT \
$DATA/train.txt \
$EXAMPLE/ilsvrc12_train_lmdb
echo"Creating val lmdb..."
GLOG_logtostderr=1$TOOLS/convert_imageset \
--resize_height=$RESIZE_HEIGHT \
--resize_width=$RESIZE_WIDTH \
--shuffle \
$VAL_DATA_ROOT \
$DATA/val.txt \
$EXAMPLE/ilsvrc12_val_lmdb
echo"Done."
最后会得到这两个文件
caffe的cifar10模型需要均值数据，可用make_imagenet_mean.sh这个脚本实现，修改下路径即可使用。
#!/usr/bin/envsh
#Compute the mean image from the imagenet training lmdb
#N.B. this is available in data/ilsvrc12
EXAMPLE=/home/caisheng/Downloads/caffe-master/examples/mytest
DATA=/home/caisheng/Downloads/caffe-master/data/my
TOOLS=/home/caisheng/Downloads/caffe-master/build/tools
$TOOLS/compute_image_mean$EXAMPLE/ilsvrc12_train_lmdb \
$DATA/imagenet_mean.binaryproto
echo"Done."
2网络定义的修改
使用caffe自己的cifar10网络，将相关文件路径修改成自己的就行。
name: "CIFAR10_quick"layer {
name: "cifar"
type: "Data"
top: "data"
top: "label"
include {
phase: TRAIN
}
transform_param {
mean_file: "/home/caisheng/Downloads/caffe-master/examples/mytest/imagenet_mean.binaryproto"
}
data_param {
source: "/home/caisheng/Downloads/caffe-master/examples/mytest/ilsvrc12_train_lmdb"
batch_size: 10
backend: LMDB
}
}
layer {
name: "cifar"
type: "Data"
top: "data"
top: "label"
include {
phase: TEST
}
transform_param {
mean_file: "/home/caisheng/Downloads/caffe-master/examples/mytest/imagenet_mean.binaryproto"
}
data_param {
source: "/home/caisheng/Downloads/caffe-master/examples/mytest/ilsvrc12_val_lmdb"
batch_size: 5
backend: LMDB
}
}
layer {
name: "conv1"
type: "Convolution"
bottom: "data"
top: "conv1"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 32
pad: 2
kernel_size: 5
stride: 1
weight_filler {
type: "gaussian"
std: 0.0001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "pool1"
type: "Pooling"
bottom: "conv1"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
}
}
layer {
name: "relu1"
type: "ReLU"
bottom: "pool1"
top: "pool1"
}
layer {
name: "conv2"
type: "Convolution"
bottom: "pool1"
top: "conv2"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 32
pad: 2
kernel_size: 5
stride: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu2"
type: "ReLU"
bottom: "conv2"
top: "conv2"
}
layer {
name: "pool2"
type: "Pooling"
bottom: "conv2"
top: "pool2"
pooling_param {
pool: AVE
kernel_size: 3
stride: 2
}
}
layer {
name: "conv3"
type: "Convolution"
bottom: "pool2"
top: "conv3"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 64
pad: 2
kernel_size: 5
stride: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu3"
type: "ReLU"
bottom: "conv3"
top: "conv3"
}
layer {
name: "pool3"
type: "Pooling"
bottom: "conv3"
top: "pool3"
pooling_param {
pool: AVE
kernel_size: 3
stride: 2
}
}
layer {
name: "ip1"
type: "InnerProduct"
bottom: "pool3"
top: "ip1"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 64
weight_filler {
type: "gaussian"
std: 0.1
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "ip2"
type: "InnerProduct"
bottom: "ip1"
top: "ip2"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2
weight_filler {
type: "gaussian"
std: 0.1
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "accuracy"
type: "Accuracy"
bottom: "ip2"
bottom: "label"
top: "accuracy"
include {
phase: TEST
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "ip2"
bottom: "label"
top: "loss"
}
由于是二分类任务，将全连接层的输出改为(num_output)2.
同样，需要修改solver文件与train-quick.sh文件中的相关路径。
#!/usr/bin/env shset -e
TOOLS=/home/caisheng/Downloads/caffe-master/build/tools
$TOOLS/caffe train \
--solver=/home/caisheng/Downloads/caffe-master/examples/mytest/cifar10_quick_solver.prototxt $@
# --solver=examples/mytest/cifar10_quick_solver.prototxt $@
# reduce learning rate by factor of 10 after 8 epochs
$TOOLS/caffe train \
--solver=/home/caisheng/Downloads/caffe-master/examples/mytest/cifar10_quick_solver.prototxt
--snapshot=/home/caisheng/Downloads/caffe-master/examples/mytest/cifar10_quick_iter_4000.solverstate $@
#--solver=examples/mytest/cifar10_quick_solver.prototxt \
#--snapshot=examples/mytest/cifar10_quick_iter_4000.solverstate $@
# reduce the learning rate after 8 epochs (4000 iters) by a factor of 10
# The train/test net protocol buffer definition
net:"/home/caisheng/Downloads/caffe-master/examples/mytest/cifar10_quick_train_test.prototxt"
# test_iter specifies how many forward passes the test should carry out.
# In the case of MNIST, we have test batch size 100 and 100 test iterations,
# covering the full 10 testing images.
test_iter: 4
# Carry out testing every 5 training iterations.
test_interval: 100
# The base learning rate, momentum and the weight decay of the network.
base_lr: 0.001
momentum: 0.9
weight_decay: 0.004
# The learning rate policy
lr_policy: "fixed"
# Display every 100 iterations
display: 100
# The maximum number of iterations
max_iter: 4000
# snapshot intermediate results
snapshot: 1000
snapshot_prefix: "/home/caisheng/Downloads/caffe-master/examples/mytest/cifar10_quick"
# solver mode: CPU or GPU
solver_mode: GPU