Remote Sensing Cross-Modal Retrieval by Deep Image-Voice Hashing

Abstract

Remote sensing image retrieval aims at searching remote sensing images of interest among immense volumes of remote sensing data, which is an enormous challenge. Direct use of voice for human–computer interaction is more convenient and intelligent. In this article, a deep image-voice hashing (DIVH) method is proposed for remote sensing image-voice retrieval. First, the whole framework is composed of the image and the voice feature learning subnetwork. Then, the hash code learning procedure will be leveraged in remote sensing image-voice retrieval to further improve the retrieval efficiency and reduce memory footprint. Hash code learning maps the deep features of images and voices into a common Hamming space. Finally, image-voice pairwise loss is proposed, which considers the similarity preservation and balance of hash codes. The similarity preserving and the balance controlling term of the loss function improve the similarity preservation from original data space to the Hamming space and the discrimination of binary code, respectively. This unified cross-modal feature and hash code learning framework significantly reduce the semantic gap between the two modal data. Experiments demonstrate that the proposed DIVH method can achieve a superior retrieval performance than other state-of-the-art remote sensing image-voice retrieval methods.

Keywords

• Convolutional neural network (CNN)
• cross-modal retrieval
• deep hashing
• hash code