OpenCV cascade classifier training

Weak classifiers using boosted cascades consist of two main stages: training and detection. The detection phase using HAAR or LBP based models is described in the Object Detection Tutorial . Here we mainly introduce the functions required to train the cascade of enhanced classifiers, including: preparing training data, performing actual model training, and visualizing training.

Table of contents

1. Training data preparation

1. Negative samples

2. Positive samples

3. Command line parameters

4. Annotation tool

2. Cascading training

1. General parameters

2. Cascading parameters

3. Enhanced classifier parameters

4. Haar-like feature parameters

3. Visual cascade classifier

1. Training data preparation

To train the augmented cascade of weak classifiers, a set of positive samples (containing the actual objects you want to detect) and a set of negative samples (containing everything you don't want to detect) are needed. The set of negative samples had to be prepared manually, while the set of positive samples was created using the opencv_createsamples application.

1. Negative samples

Negative samples are taken from arbitrary images that do not contain the object to be detected. The images from which samples are generated should be placed in separate files, one image path per line. Negative sample images are also known as background samples or background images.

Images depicted may be of different sizes. However, each image should be equal to or larger than the desired training window size.

Directory Structure:

/img
  img1.jpg
  img2.jpg
bg.txt

bg.txt mainly specifies the image path:

img/img1.jpg
img/img2.jpg

2. Positive samples

Positive samples are created by the opencv_createsamples application. The augmentation process uses them to define what the model is trying to find about objects of interest to us. Two methods of generating positive datasets are supported, and the second method is recommended.

1. Generate a sequence of positive images from a single positive image.
2. Provide all positive samples yourself, just use the tool to cut them out, resize them and put them into the binary format required by opencv.

3. Command line parameters

The command line includes positive sample path, negative sample path, output path, sample width and height, etc., as follows:

• -vec <vec_file_name>: training result output path;
• -img <image_file_name>: positive sample path;
• -bg <background_file_name>: Negative sample path;
• -num <number_of_samples>: the number of positive samples to generate;
• -bgcolor <background_color>: background color, ie transparent color;
• -inv: if specified, colors will be inverted;
• -randinv: if specified, colors will be randomly inverted;
• -maxxangle <max_x_rotation_angle>: The maximum rotation angle towards the x-axis, in radians;
• -maxxangle <max_y_rotation_angle>: The maximum rotation angle towards the y-axis, in radians;
• -maxxangle <max_z_rotation_angle>: The maximum rotation angle toward the z-axis, in radians;
• -show: debug options;
• -w: the width of the output sample;
• -h: the height of the output sample;

When running opencv_createsamples in this way, the following process is used to create sample object instances: The given source image is randomly rotated around all three axes. The selected angle is affected by -maxxangle, -maxyangleand -maxzangle限制. White noise is added to the intensity of the foreground. If specified -inv, foreground pixel intensities are inverted. Finally, place the obtained image on any background in the background description file, adjust the width and height to sum -wand -h，store it in the path specified by the command line -vec.

Positive samples can also be obtained from a collection of previously labeled images, which is required when building a stable object model. This collection is described by a text file similar to the background description file. Each line of the file corresponds to an image. The first element of the line is the filename, then the number of object annotations, then numbers describing the coordinates (x, y, width, height) of the object's bounding rectangle, described in the file below.

Directory Structure:

/img
  img1.jpg
  img2.jpg
info.dat

Image 1 in info.dat contains a rectangle coordinate, and image 2 contains 2 rectangle coordinates, as follows:

img/img1.jpg 1 140 100 45 45
img/img2.jpg 2 100 200 50 50 50 30 25 25

4. Annotation tool

After OpenCV3.x, an open source annotation tool -info for generating files is provided, and the relevant parameters are as follows:

• --annotations (required): The path to the txt file annotations, where the annotations are stored;
• --images (required): path to folder containing images;
• --maxWindowHeight: If the height of the input image is greater than the height limited here, the image will be resized;
• --resizeFactor: The factor used to resize the image;

An example command line is as follows:

opencv_annotation --annotations=/xxx/annotations/file.txt --images=/xxx/image/

2. Cascading training

After the positive and negative samples are prepared, the weak classifiers are trained in cascade. The opencv_traincascade command is divided according to its functional purpose.

1. General parameters

Common parameters include positive and negative sample path, number of positive and negative samples, classifier output path, and number of training cascade stages, as follows:

• -data <cascade_dir_name>: The output path of the classifier, the folder needs to be created in advance;
• -vec <vec_file_name>: positive sample file path;
• -bg <background_file_name>: Negative sample file path;
• -numStages <number_of_stages>: The number of stages of cascade training;
• -precalcValBufSize：buffer size of precomputed eigenvalues ​​in Mb;
• -precalcIdxBufSize: buffer size (in Mb) for precomputed feature indices;
• -numThreads <max_number_of_threads>: The maximum number of threads used for training;
• -acceptanceRatioBreakValue: Used to determine how accurate the model should keep learning and when to stop. The default is -1, which disables this feature.

2. Cascading parameters

Cascading parameters include stage type, feature type, sample width and height, as follows:

• -stageType <BOOST(默认)>: stage type;
• -featureType<{HAAR(默认), LBP}>: feature type: HAAR (Haar-like feature), LBP (local binary pattern);
• -w <sampleWidth>: the width of the training sample (in pixels);
• -h <sampleHeight>: the height of the training sample (in pixels);

3. Enhanced classifier parameters

Enhanced classifier parameters include classifier type, minimum hit rate, and maximum error rate, as follows:

• -bt <{DAB, RAB, LB, GAB(默认)}>: The type of enhanced classifier: DAB (Discrete AdaBoost), RAB (Real AdaBoost), LB (LogitBoost), GAB (Gentle AdaBoost);
• -minHitRate: The minimum expected hit rate for each stage of the classifier;
• -maxFalseAlarmRate: The maximum expected error rate of each stage of the classifier;
• -weightTrimRate: Whether to use correction weight, the recommended value is 0.95;
• -maxDepth <max_depth_of_weak_tree>: The maximum depth of the weak tree, the recommended value is 1;
• -maxWeakCount <max_weak_tree_count>: the maximum number of weak trees per cascade stage;

4. Haar-like feature parameters

Haar-like feature parameters mainly specify the feature set type, as follows:

• -mode <BASIC(默认) | CORE | ALL>: Select the type of Haar feature set used in training. Among them, BASIC only uses the upright function, while ALL uses the upright and 45 degree rotation function set;

3. Visual cascade classifier

Visual cascade classifiers are used to see a real-time view of the training data during training. OpenCV provides the opencv_visualisation application, and the related commands are as follows:

• --image (required): the path of the image model;
• --model (required): the path of the training model;
• --data (optional): If a data folder created in advance is provided, the results of each stage will be stored;

An example command line is as follows:

opencv_visualisation --image=/data/object.png --model=/data/model.xml --data=/data/result/