Fine-grained Semantic Reasoning Based Cross-media Dual-way Adversarial Hashing Learning Model

Abstract

Cross-media hashing has received extensive attention in cross-media searching tasks due to its superior searching efficiency and low storage cost.However,existing methods cannot adequately preserve the high-level semantic relevance and multi-label of multi-media data.In order to solve the above problems,this paper proposes a fine-grained semantic reasoning based cross-media dual-way adversarial hashing learning model(SDAH),which generates compact and consistent cross-media unified efficient hash semantic representations by maximizing fine-grained semantic associations between different medias.First,a fine-grained cross-media semantic association learning and inference method based on the cross-media collaborative attention mechanism is proposed.The cross-media attention mechanism collaboratively learns the fine-grained implicit semantic associations of images and texts,and obtains the salient semantic inference features of images and texts.Then,a cross-media dual-way adversarial hashing network is established to jointly learn the intra-modality and inter-modality semantic similarity constraints,and better to align the semantic distributions of different media hash codes through a two-way adversarial learning mechanism,which generates higher-quality and more discriminative cross-media unified hash representation,facilitates the process of cross-media semantic fusion and improves the cross-media searching performance.Experimental results compared with existing methods on two public datasets verify the performance superiority of the proposed method in various cross-media search scenarios.

Keywords

• semantic reasoning|hash learning|cross-media search|adversarial learning|cross-media semantic fusion