Learning DALTS for cross-modal retrieval

Abstract

Cross-modal retrieval has been recently proposed to find an appropriate subspace, where the similarity across different modalities such as image and text can be directly measured. In this study, different from most existing works, the authors propose a novel model for cross-modal retrieval based on a domain-adaptive limited text space (DALTS) rather than a common space or an image space. Experimental results on three widely used datasets, Flickr8K, Flickr30K and Microsoft Common Objects in Context (MSCOCO), show that the proposed method, dubbed DALTS, is able to learn superior text space features which can effectively capture the necessary information for cross-modal retrieval. Meanwhile, DALTS achieves promising improvements in accuracy for cross-modal retrieval compared with the current state-of-the-art methods.

Keywords

• information retrieval
• text analysis
• natural language processing
• image segmentation
• image retrieval
• recurrent neural nets
• cross-modal retrieval
• DALTS
• domain-adaptive limited text space
• image space
• Flickr8K
• Flickr30K
• MSCOCO
• text space features