Cross-Modal Retrieval by Class Information and Listwise Ranking

Abstract

Cross-modal retrieval has attracted significant attention due to the increasing use of multi-modal data. A major challenge for cross-modal retrieval is the modal gap. To cope with the heterogeneity, common subspace learning method is proposed. However, most of them mainly focus on relevant or irrelevant information, and do not consider the relevant and irrelevant information simultaneously. In addition, there are many pairwise methods for cross-modal retrieval, but they do not consider the internal dependencies between the doc pairs corresponding to the same query and do not make full use of the structure between the samples. To take full account of the intra-class and inter-class relationships between samples, the cross-modal retrieval by listwise ranking and class information (C2MLR2) is proposed, which maximizes the similarity of positive samples to the anchor and meanwhile minimizes the similarity of the negative samples to the anchor via listwise ranking. Experimental results verify the effectiveness of the algorithm in cross-modal retrieval.

Keywords

• relevant information
• irrelevant information
• intra-class
• inter-class
• listwise ranking