Original paper: AutoAugment: Learning Augmentation Policies from Data

Paper address: http://export.arxiv.org/abs/1805.09501

The above address downloads the latest version (v3) revised in April 2019

This article is an automatic data enhancement method proposed by the Google Brain team. The reinforcement learning algorithm is used to find the most suitable enhancement strategy for each data set, and the migration learning of the pre-trained model can transfer the obtained data enhancement strategy to different data sets. .

Specific search process:

1. Define N data enhancement methods, which are the most basic data enhancement methods, such as rotation, translation, flipping, etc. Each data enhancement method has its trigger probability and magnitude value (such as the angle range of rotation) .

2. Construct 5 sub-strategy methods (there are 5 sub-strategies in the paper, or you can manually define any number of sub-strategies for comparison and selection), and each sub-strategy method is composed of 2 data enhancement methods (same as above, but specified as 2 in the paper).

3. Use these sub-strategies for training and verification in a small data set, and select the data enhancement scheme with the highest accuracy.

Search algorithm details:

The search algorithm used is reinforcement learning. The algorithm has two parts: the RNN controller and the training algorithm PPO.

At each step, the controller makes a decision through softmax, and the decision is then input to the next step. There are 30 decisions in total, and 5 sub-strategies can be predicted. Each sub-strategy includes two operations, and each operation requires two operation parameters, which are probability and usage.

The controller is trained with a reward signal that indicates how much the policy improves model generalization. In experiments, we set up a validation set to measure generalization. Each model is trained on data augmented with 5 sub-strategies: For each sample in the mini-batch, one of the 5 sub-strategies is randomly selected to augment the image. The model is measured with accuracy on the validation set, which also serves as a reward signal to train the controller.

Comments:

This paper is the pioneering work of automatic data enhancement. The disadvantage is that the training is too slow, and the search for the optimal enhancement strategy is defined as a discrete search problem. There are also some follow-up work to improve it, such as fast augment

Original paper: Learning Data Augmentation Strategies for Object Detection

This paper was also published by the Google Brain team, which is almost the same team. As can be seen from the title of the article, it is actually the extension of automatic data enhancement to the target detection task. The author found that image classification is a simple cv task, the data enhancement method is different from the data enhancement of the target detection task, and the optimal data enhancement strategy is also different, so the author wants to explore whether automatic data enhancement can be realized.

The auto augmentation in mmdetection is also implemented in this paper.

Implementation details:

The specific idea is similar to the previous article. Various data enhancement methods (note that because it is target detection, not only simple image rotation, etc., but also some operation transformations for bbox) are brought with parameters (probability, amplitude values) are combined as discrete values into sub-policies, which are searched with reinforcement learning and an auxiliary RNN controller.

Noteworthy conclusions:

The paper found that for the target detection task, using the automatic data enhancement method to search for the optimal data enhancement strategy sequence, the performance on a small number of data sets is better than that on a large number of data sets, and the detection accuracy of small targets is improved. Compared with the improvement of the big target, the improvement of mAP0.75 is greater than that of mAP0.5 (this can be understood as bbox is closer to the actual gt).

Furthermore, the authors find that other successful regularization techniques are not beneficial if applied in conjunction with learned data augmentation strategies. They ran several experiments with typos, Manifold Mixup, and droppers. For all methods, we found that they neither help nor hurt model performance. This is an interesting result because the proposed method independently outperforms these regularization methods, but it is clear that these regularization methods are not needed when applying a learned data augmentation strategy.

AutoAugment introduction and paper analysis