Click on the card below to follow the " CVer " public account
AI/CV heavy-duty information, delivered as soon as possible
Click to enter -> [Remote Sensing Imagery and Target Detection] Communication Group
Reply in the background of CVer WeChat public account: MUS, you can download the pdf of this paper
MUS-CDB: Mixed Uncertainty Sampling with Class Distribution Balancing for Active Annotation in Aerial Object Detection
Article link:
https://ieeexplore.ieee.org/document/10158738
Home page: https://dongl-group.github.io
introduction:
Existing remote sensing object detection models usually rely on a large amount of labeled training data. However, due to the high density of objects in remote sensing scenes, the time and money required for manual labeling are very expensive. Active learning effectively reduces data annotation costs by selectively querying informative and representative unlabeled samples. However, existing active learning methods are mainly suitable for class-balanced settings and image query-based general object detection tasks, while in remote sensing object detection scenarios, due to the long-tailed class distribution and dense small objects, these methods are not very applicable. As shown in Figure 1, compared with the distribution of objects in common scenes, the targets in aerial remote sensing images are usually smaller, fuzzy, and densely distributed in complex backgrounds.
In this paper, we propose a novel active learning method for remote sensing target detection, aiming to effectively reduce costs. Specifically, both object-level and image-level informativeness are considered in object sampling to avoid redundant and short-sighted queries. Furthermore, an easy-to-use class balancing criterion is incorporated to support minority class objects and alleviate the long-tail class distribution problem in model training. We further design a training loss to mine latent knowledge in unlabeled image regions.
method:
Remote sensing object detection using active learning aims to train a well 
- to reduce the labeling cost 
. The problem is defined by three sets of data: a small fully labeled set used to initialize the model 
, a large unlabeled set used for data selection 
, and a partially labeled set sampled by active learning methods 
. In order to train a well-performing detector with the minimum labeling cost 
, we use the sampling 
function to select the most informative samples for labeling, and the labeled pictures are added 
. We designed two modules for cost-effective sampling: Mixed Uncertainty Sampling module (MUS) and Class Distribution Balanced Sampling (CDB). The overall framework of the method is shown in Figure 2.
Mixed uncertainty sampling:
Existing object-based sampling methods mainly consider the information of the prediction box itself, namely category uncertainty or regression uncertainty, but ignore the spatial information and semantic structure of the image. To address this issue, we propose to consider image and object uncertainty, i.e., combine global and local information for a more comprehensive data assessment.
Regarding image uncertainty, if there are many predicted objects in an image with high uncertainty, then that image should be preferentially selected for sampling. To do this, we evaluate and aggregate the uncertainty values of model predictions to represent the uncertainty value of the entire image. Specifically, for a given image 
, the expression for image uncertainty 
is as follows:
where 
represents the number of elements in the set and 
is the score threshold. The image uncertainty value 
is obtained by calculating the average confidence of the bounding boxes in the image. Only bounding boxes with confidence greater than the threshold 
are used to calculate the average confidence. 
The value will be higher when there are many lower confidence predicted bounding boxes in the image . This is because the images contain indistinguishable objects, resulting in inconsistent and low-confidence predictions. Therefore, 
images with higher values are more likely to contain useful information about rare patterns and are therefore more suitable for selection.
Regarding object uncertainty, in order to consider object-level information in queries, we use entropy to evaluate the uncertainty of each predicted bounding box. Specifically, the object uncertainty 
is calculated as follows:
where 
is the predicted probability of the category of the bounding box 
in the image.
Next, we combine image uncertainty 
and object uncertainty 
to obtain the final object information score 
.
Class distribution balanced sampling:
Remote sensing data suffers from class imbalance, where rare classes severely harm model performance. To address this problem, we propose a sampling method that emphasizes low-frequency categories during active querying. Specifically, we first count the distribution of classes in the labeled dataset, and then identify the rare classes in the labeled dataset. Let represent the number of objects corresponding to 
the category , where . Our goal is to query rare category objects more often during the sampling phase by imposing a preference inversely proportional to each category. Sampling preference is calculated as follows:
First, based on the ak value, we calculate the distribution probability of each category in the labeled set. Then, we take its reciprocal to get the category weight βk, which is used for weight adjustment during the sampling process. Next, we use the Softmax function to calculate the expected class distribution during the sampling period. In this way, we are able to set preferences for different categories 
and selectively query rare category objects during the selection phase to improve the performance and accuracy of the model.
Process partially labeled images:
To deal with the situation during model training that some datasets are fully labeled and other datasets of images are only partially labeled, we employ different training loss functions for these two sets. For the fully labeled dataset, we follow the default training loss function of the detector; while for the partially labeled dataset, we adopt a custom loss function to effectively mine the latent knowledge of the unlabeled regions in the image.
Specifically, partially labeled images can introduce noise to the negative loss in classification loss when the model is trained, because some objects in the image may not be labeled and regarded as negative samples. To solve this problem, we propose an adaptive weight loss function to handle the negative sample loss in classification loss. This method adjusts the corresponding classification loss weight of each negative sample based on its predicted background score. This method can effectively suppress the model's classification loss for negative samples (usually foreground objects) with low background scores. 
The definition is as follows:
Contains classification loss (first two items) and box regression loss (last item). Among them, i and j are the indices of image and region proposals in a mini-batch, and W represents the number of region proposals participating in training. 
and 
are indicator functions that indicate whether the image is partially tagged or fully tagged. 
Used to indicate whether the region proposal is a positive sample (i.e. contains an object). To achieve robust learning, parameters are introduced 
to reduce the classification loss weight of background objects.
experiment:
1. Comparison with other active learning methods
We study four remote sensing detectors (including two single-stage remote sensing detectors: KLD and SASM, and two dual-stage remote sensing detectors: ReDet and Oriented R-CNN) and two datasets (DOTA-v1.0 and DOTA -v2.0). We use mAP as a comparison metric. The experimental results are shown in Table I. The effectiveness and versatility of the proposed MUS-CDB method are demonstrated through experiments on multiple detectors. This method can be easily integrated into various object detection frameworks and helps improve object detection model performance in different applications.
Entropy sampling only considers target-level information during the sampling process, while hybrid uncertainty sampling considers target-level and image-level information comprehensively. To verify the effectiveness of hybrid sampling, we performed a performance comparison of the two sampling methods.
2. Ablation experiment
To demonstrate the effectiveness of our proposed two sampling modules, we conducted the following ablation experiments. It can be seen that both modules of Uncertainty Sampling (MUS) and Class Distribution Balanced Sampling (DUS) can effectively improve model performance. The two-stage sampling combining the two can better balance the diversity and representativeness of the sampling results.
We also performed ablation experiments to prove the effectiveness of the adaptive loss function. (1) represents using the default loss function, (2) represents using the proposed improved loss.
in conclusion:
In this paper, we propose an object-based active learning method named MUS-CDB, which aims to alleviate the huge burden posed by the annotation of remote sensing object detection data. We design a hybrid uncertainty sampling module based on images and objects during the sampling process to select the most informative instances for annotation. Considering the long-tail problem in remote sensing image datasets, we introduce a category preference strategy in the sampling process to promote the diversity of selected objects. Furthermore, we propose an efficient training method on partially labeled data to fully exploit the knowledge gained from active queries.
Reply in the background of CVer WeChat public account: MUS, you can download the pdf of this paper
Click to enter -> [Remote Sensing Imagery and Target Detection] Communication Group
ICCV/CVPR 2023 paper and code download
Backstage reply: CVPR2023, you can download the collection of CVPR 2023 papers and code open source papers
后台回复：ICCV2023，即可下载ICCV 2023论文和代码开源的论文合集目标检测和遥感图像交流群成立
扫描下方二维码，或者添加微信：CVer333，即可添加CVer小助手微信，便可申请加入CVer-目标检测或者遥感图像 微信交流群。另外其他垂直方向已涵盖：目标检测、图像分割、目标跟踪、人脸检测&识别、OCR、姿态估计、超分辨率、SLAM、医疗影像、Re-ID、GAN、NAS、深度估计、自动驾驶、强化学习、车道线检测、模型剪枝&压缩、去噪、去雾、去雨、风格迁移、遥感图像、行为识别、视频理解、图像融合、图像检索、论文投稿&交流、PyTorch、TensorFlow和Transformer、NeRF等。
一定要备注：研究方向+地点+学校/公司+昵称（如目标检测或者遥感图像+上海+上交+卡卡），根据格式备注，可更快被通过且邀请进群
▲扫码或加微信号: CVer333，进交流群
CVer计算机视觉（知识星球）来了！想要了解最新最快最好的CV/DL/AI论文速递、优质实战项目、AI行业前沿、从入门到精通学习教程等资料，欢迎扫描下方二维码，加入CVer计算机视觉，已汇集数千人！
▲扫码进星球
▲点击上方卡片，关注CVer公众号整理不易，请点赞和在看