Check the windows system bloggers article: https://blog.csdn.net/m0_43505377/article/details/90613529
First, the environment configuration;
⑴ Anaconda (time to install, but the tutorial after tutorial, please use the system environment ):
Anaconda is an open source package, environment manager can be used to install different versions of packages and their dependence on the same machine, and can be switched between different environments
https://repo.anaconda.com/ archive / Anaconda3-2019.03-MacOSX-x86_64.pkg
this is Linux64-bit download python3.7 version
other versions:
https://www.anaconda.com/distribution/#download-section
please download the official website (corresponding to its own operating system Download python3.7 version);
can create a tensorflow (python environment select 3.6 or 3.7) environment, you can use the foundation base (default python3.7) environment
to create a new environment
conda create -n tensorflow python=3.6
Conda simple command:
viewing environment within the system:
conda info --e
Creation Environment
conda create -n xxx python=2.7// xxx created in the name of the environment python27 with the
environment activation source activate xxx
environment exit deactivate xxx
delete environmentconda remove -n xxx --all
conda install package//安装包
conda list//查看已安装的包
conda update package//更新包
conda remove package//移除包
The library needs to have: tensorflow (GPU or CPU version), pillow, lxml, matplotlib, jupyter, your own pip install.
Before installing the packages we need to activate the environment
source activate tensorflow(需要激活的环境）
pip install (所需要的包）
⑵ Object Detection API to download Tensorflow:
https://github.com/tensorflow/models (GitHub)
Create a new folder for storing packages:
mkdir pakege
cd pakeage
Then cloned git package
without git installed, install git:
sudo apt-get install git
Cloning package
git clone https://github.com/tensorflow/models （需要git的地址）
It is based on open source framework tensorflow structure, easy to build, train, and deploy target detection model, the download on GitHub. Due to restrictions on the use of domestic network download speed is very slow, I can choose to share the Baidu cloud connection to download:
Link: https://pan.baidu.com/s/1zp22K5zHVRyrpoQqB5xi1g
extraction code: 2xiu
After you copy the contents of this open Baidu network disk phone App, the operation more convenient oh
then ptf transmission:
detailed bloggers blog: https://blog.csdn.net/m0_43505377/article/details/90668779
⑶ download Protoc:
1, Google Protocol Buffer (referred Protobuf) is Google mixed internal standard language of data
downloaded in the GitHub
https://github.com/protocolbuffers/protobuf
download version corresponding to the own system, such as: protoc-3.7.1-win64.zip
clone protoc
git clone https://github.com/protocolbuffers/protobuf
2, find Tensorflow object detection API directory, the command line cd to the models \ research file
A: Input activate activation anconda environment
如： source activate tensorflow
B: Continue to enter:
protoc object_detection/protos/*.proto --python_out=.
3, in the new system variable called: "PYTHONPATH" variable, the complete catalog Tensorflow object research / and research in the detection API directory / slim two folders will be added. (Note separated by a semicolon) at the same time ... \ models-master \ research \ object_detection added to the system path variable (its path)
Linux directly execute:
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
4, slim loading module folder:
A: cd switched to the research / slim folder (Method: The input at which partition: E: continue typing: cd path) The full path to the path of research / slim folder .
Input:
python setup.py install
5, the installation pycocotools:
A: Download COCO on GitHub
https://github.com/philferriere/cocoapi
git clone https://github.com/philferriere/cocoapi
PythonAPI cd to the directory, enter:
python setup.py install
If the error can not find vcversall.bat, install install Visual Studio2015
Second, the training preparation:
⑴ pictures marked
before the start of training we need to make the required training, said tagging pictures, we recommend using labelImg labeling software (download link)
HTTPS: // github.com/tzutalin/labelImg/releases
Installation labelImg
sudo apt-get install pyqt5-dev-tools
sudo pip install -r requirements / requirements-linux-python3.txt
run
python3 labelImg.py
或者： python3 labelImg.py [IMAGE_PATH] [预先定义的类文件]
Use LabelImg this small software, manual tagging.
Open dir Open Folder
Change save dir generated file save folder
tagging method: marquee region of interest shortcuts w
A switched to the left image
D is switched to the right image
⑵ generate csv table:
directly after the message using the modified codes
"""
Created on Tue Jan 16 00:52:02 2018
@author: Xiang Guo
将文件夹内所有XML文件的信息记录到CSV文件中
"""
import os
import glob
import pandas as pd
import xml.etree.ElementTree as ET
os.chdir(' #输入图片文件夹 ')
path = '#cvs表格输出文件夹 '
def xml_to_csv(path):
xml_list = []
for xml_file in glob.glob(path + '/*.xml'):
tree = ET.parse(xml_file)
root = tree.getroot()
for member in root.findall('object'):
value = (root.find('filename').text,
int(root.find('size')[0].text),
int(root.find('size')[1].text),
member[0].text,
int(member[4][0].text),
int(member[4][1].text),
int(member[4][2].text),
int(member[4][3].text)
)
xml_list.append(value)
column_name = ['filename', 'width', 'height', 'class', 'xmin', 'ymin', 'xmax', 'ymax']
xml_df = pd.DataFrame(xml_list, columns=column_name)
return xml_df
def main():
image_path = path
xml_df = xml_to_csv(image_path)
xml_df.to_csv('tv_vehicle_labels.csv', index=None)
print('Successfully converted xml to csv.')
main()
⑶ because this model only supports input TFRecord format, we need a table to CSV data into this format.
Referring to modify the code
cd （位置）
Create a new file in the generate_tfrecord.py research \ object_detection, copy the code, modify the prompts
touch generate_tfrecord.py
Conversion Code:
# -*- coding: utf-8 -*-
"""
Created on Tue Jan 16 01:04:55 2018
@author: Xiang Guo
由CSV文件生成TFRecord文件
"""
"""
Usage:
# From tensorflow/models/
# Create train data:
python generate_tfrecord.py --csv_input=data/tv_vehicle_labels.csv --output_path=train.record
# Create test data:
python generate_tfrecord.py --csv_input=data/test_labels.csv --output_path=test.record
"""
import os
import io
import pandas as pd
import tensorflow as tf
from PIL import Image
from research.object_detection.utils import dataset_util
from collections import namedtuple, OrderedDict
os.chdir('#工作目录就是object_detection的位置')
#如：...\\models-master\\research\\object_detection\\
flags = tf.app.flags
flags.DEFINE_string('csv_input', '', 'Path to the CSV input')
flags.DEFINE_string('output_path', '', 'Path to output TFRecord')
FLAGS = flags.FLAGS
# TO-DO replace this with label map
# 注意将对应的label改成自己的类别！！！！！！！！！！
def class_text_to_int(row_label):
if row_label == 'cherry': #改为自己的标签
return 1 #返回的数值从1 开始
elif row_label == "Areca":
return 2
elif row_label == "xiakucao":
return 3
elif row_label == "mudanpi":
return 4
else:
None
def split(df, group):
data = namedtuple('data', ['filename', 'object'])
gb = df.groupby(group)
return [data(filename, gb.get_group(x)) for filename, x in zip(gb.groups.keys(), gb.groups)]
def create_tf_example(group, path):
with tf.gfile.GFile(os.path.join(path, '{}'.format(group.filename)), 'rb') as fid:
encoded_jpg = fid.read()
encoded_jpg_io = io.BytesIO(encoded_jpg)
image = Image.open(encoded_jpg_io)
width, height = image.size
filename = group.filename.encode('utf8')
image_format = b'jpg'
xmins = []
xmaxs = []
ymins = []
ymaxs = []
classes_text = []
classes = []
for index, row in group.object.iterrows():
xmins.append(row['xmin'] / width)
xmaxs.append(row['xmax'] / width)
ymins.append(row['ymin'] / height)
ymaxs.append(row['ymax'] / height)
classes_text.append(row['class'].encode('utf8'))
classes.append(class_text_to_int(row['class']))
tf_example = tf.train.Example(features=tf.train.Features(feature={
'image/height': dataset_util.int64_feature(height),
'image/width': dataset_util.int64_feature(width),
'image/filename': dataset_util.bytes_feature(filename),
'image/source_id': dataset_util.bytes_feature(filename),
'image/encoded': dataset_util.bytes_feature(encoded_jpg),
'image/format': dataset_util.bytes_feature(image_format),
'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),
'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),
'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),
'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),
'image/object/class/text': dataset_util.bytes_list_feature(classes_text),
'image/object/class/label': dataset_util.int64_list_feature(classes),
}))
return tf_example
def main(_):
writer = tf.python_io.TFRecordWriter(FLAGS.output_path)
path = os.path.join(os.getcwd(), 'images')
examples = pd.read_csv(FLAGS.csv_input)
grouped = split(examples, 'filename')
for group in grouped:
tf_example = create_tf_example(group, path)
writer.write(tf_example.SerializeToString())
writer.close()
output_path = os.path.join(os.getcwd(), FLAGS.output_path)
print('Successfully created the TFRecords: {}'.format(output_path))
if __name__ == '__main__':
tf.app.run()
In object_detection images folder create a new folder, you will need to convert each image and label information into xml.
Switch to open cmd tensorflow environment (environmental tensorflow installed)
then switched to the switching position to: object_detection folder
生成测试集的命令：
python generate_tfrecord.py --csv_input=data/tv_vehicle_labels.csv
代码文件名 CVS文件路径及文件名
生成训练集:
python generate_tfrecord.py --csv_input=data/test_labels.csv
Third, start training:
download the configuration file address:
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md
1, the configuration file trained
to ssd_mobilenet_v1_coco.config example, in object_dection file under folder, unzip the downloaded file, such as: t_v1_coco_2017_11_17.tar.gz,
the ssd_mobilenet_v1_coco.config placed in the training folder, open (I use the sublime 3) with a text editor, proceed as follows:
1, which PATH_TO_BE_CONFIGURED search , the corresponding path to its own path, be careful not to test with the train backwards;
2, according to the actual situation will change num_classes
example :( after the code annotated)
model {
ssd {
num_classes: 4 #你的多少种标签就更改为多少
box_coder {
faster_rcnn_box_coder {
y_scale: 10.0
x_scale: 10.0
height_scale: 5.0
width_scale: 5.0
}
}
matcher {
argmax_matcher {
matched_threshold: 0.5
unmatched_threshold: 0.5
ignore_thresholds: false
negatives_lower_than_unmatched: true
force_match_for_each_row: true
}
}
similarity_calculator {
iou_similarity {
}
}
anchor_generator {
ssd_anchor_generator {
num_layers: 6
min_scale: 0.2
max_scale: 0.95
aspect_ratios: 1.0
aspect_ratios: 2.0
aspect_ratios: 0.5
aspect_ratios: 3.0
aspect_ratios: 0.3333
}
}
image_resizer {
fixed_shape_resizer {
height: 300
width: 300
}
}
box_predictor {
convolutional_box_predictor {
min_depth: 0
max_depth: 0
num_layers_before_predictor: 0
use_dropout: false
dropout_keep_probability: 0.8
kernel_size: 1
box_code_size: 4
apply_sigmoid_to_scores: false
conv_hyperparams {
activation: RELU_6,
regularizer {
l2_regularizer {
weight: 0.00004
}
}
initializer {
truncated_normal_initializer {
stddev: 0.03
mean: 0.0
}
}
batch_norm {
train: true,
scale: true,
center: true,
decay: 0.9997,
epsilon: 0.001,
}
}
}
}
feature_extractor {
type: 'ssd_mobilenet_v1'
min_depth: 16
depth_multiplier: 1.0
conv_hyperparams {
activation: RELU_6,
regularizer {
l2_regularizer {
weight: 0.00004
}
}
initializer {
truncated_normal_initializer {
stddev: 0.03
mean: 0.0
}
}
batch_norm {
train: true,
scale: true,
center: true,
decay: 0.9997,
epsilon: 0.001,
}
}
}
loss {
classification_loss {
weighted_sigmoid {
}
}
localization_loss {
weighted_smooth_l1 {
}
}
hard_example_miner {
num_hard_examples: 3000
iou_threshold: 0.99
loss_type: CLASSIFICATION
max_negatives_per_positive: 3
min_negatives_per_image: 0
}
classification_weight: 1.0
localization_weight: 1.0
}
normalize_loss_by_num_matches: true
post_processing {
batch_non_max_suppression {
score_threshold: 1e-8
iou_threshold: 0.6
max_detections_per_class: 100
max_total_detections: 100
}
score_converter: SIGMOID
}
}
}
train_config: {
batch_size: 8
optimizer {
rms_prop_optimizer: {
learning_rate: {
exponential_decay_learning_rate {
initial_learning_rate: 0.004
decay_steps: 800720
decay_factor: 0.95
}
}
momentum_optimizer_value: 0.9
decay: 0.9
epsilon: 1.0
}
}
#fine_tune_checkpoint: "E:\\asdas\\models-master\\research\\object_detection\\training\\model.ckpt"
#from_detection_checkpoint: true
num_steps: 200000
data_augmentation_options {
random_horizontal_flip {
}
}
data_augmentation_options {
ssd_random_crop {
}
}
}
train_input_reader: {
tf_record_input_reader {
input_path: " " **#更改为自己的训练集路径**
}
label_map_path: " " **#对应的pbtxt文件**
}
eval_config: {
num_examples: 8000
# Note: The below line limits the evaluation process to 10 evaluations.
# Remove the below line to evaluate indefinitely.
max_evals: 10
}
eval_input_reader: {
tf_record_input_reader {
input_path: " " **#更改为自己的测试集路径**
}
label_map_path: " " **#对应的pbtxt文件**
shuffle: false
num_readers: 1
}
config file label_map_path: "data / tv_vehicle_detection.pbtxt" need to be consistent.
At this point in the corresponding directory (/ data), create a text file (you can copy a file suffix, like other names, and then modify with a text editor to open the software), we write the label, my example is two, when id number is consistent with the previous note to create a CSV file, starting at 1.
touch tv_vehicle_detection.pbtxt
item {
id: 1
name: 'tv'
}
item {
id: 2
name: 'vehicle'
}
2, start training:
cd switched to the environment of the installation tensorflow.
Locate the models \ under research \ object_detection folder, run the following command:
# From the tensorflow/models/research/ directory
python object_detection/model_main.py \
--pipeline_config_path=object_detection/training/ssd_mobilenet_v1_coco.config \
--model_dir=object_detection/training \
--num_train_steps=50000 \
--num_eval_steps=2000 \
--alsologtostderr
pipeline_config_path = (ssd_mobilenet_v1_coco.config path)
num_train_steps = (number of training rounds)
num_eval_steps = (how many times a test)
model_dir = (model save location)
Not every print after the start of training information
using the recommended open another command line, activate anconda environment
run the following code opens tensorboard:
tensorboard --logdir=(模型保存位置) --host=127.0.0.1
Copy the URL displayed after a successful operation to open the browser to open tensorboard
