Multiscale Image Splitting Based Feature Enhancement and Instance Difficulty Aware Training for Weakly Supervised Object Detection in Remote Sensing Images

Abstract

Weakly supervised object detection (WSOD) has a great practical value in remote sensing image (RSI) interpretation because the instance-level annotations are not required. The multiple instance learning based methods are mainstream, and two problems should be addressed. First of all, the majority of methods usually detect discriminative parts rather than the whole object. Secondly, the quantity of easy instances is much greater than that of hard instances, which restricts the improvement of WSOD methods. To address the first problem, a multiscale image splitting based feature enhancement (MSFE) module is proposed. The MSFE module splits the input RSI in multiple scales, afterwards, the spatial attention maps (SAMs) are generated from the feature maps of each proposal corresponding to different splitting scales, and are used to calculate the maximum spatial attention map (MSAM). Each SAM is required to approach MSAM, which enforces the MSFE module to learn the feature maps which can highlight the whole object for each positive proposal. To address the second problem, an instance difficulty aware training (IDAT) strategy is proposed. The difficulty of each instance can be quantitatively measured, and is used as the weight of each instance in the training loss. Consequently, the hard instances will be focused in the training process. The ablation study demonstrates the validity of MSFE module and IDAT strategy. The comparisons with nine advanced methods on two RSI benchmarks further validate the overall effectiveness of our method.

Keywords

• Instance difficulty aware training strategy
• multiscale image splitting based feature enhancement (MSFE)
• remote sensing image (RSI)
• weakly supervised object detection (WSOD)