Feature Fusion for Image Retrieval With Adaptive Bitrate Allocation and Hard Negative Mining

Abstract

By combining Convolutional Neural Network (CNN) descriptor and Compact Descriptors for Visual Search (CDVS), the visual search performance can be boosted. However, some redundancies still exist in the CDVS representation and the hard negative mining is not very accurate when training CNN embeddings. In this paper, we propose a high performance image retrieval scheme based on descriptor fusion. In detail, we first propose a more compact CDVS descriptor database building scheme through bitrate allocation, which can reduce information redundancy and boost image retrieval performance. We then propose a highly accurate CDVS-guided hard negative mining scheme when training CNN embeddings. In the hard negative selection, the CDVS descriptor and the CNN embedding are adaptively weighted together to achieve more precise decisions. Finally, the retrieval result is further refined through CDVS local descriptor matching by removing the irrelevant targets from the top positions. Extensive experimental results show that the proposed method outperforms the recent hybrid method and several other anchors remarkably, and produces better visual search performance. Codes and some models are available at https://github.com/WendyDong/ImageRetrieval_DF_CDVS.

Keywords

• Image retrieval
• CDVS
• CNN
• hard negative mining
• deep learning