Overview of Natural Language Video Localization

Abstract

Natural language video localization(NLVL),which aims to locate a target moment from a video that semantically corresponds to a text query,is a novel and challenging task.Different from the task of temporal action localization,NLVL is more flexible without restrictions from predefined action categories.Meanwhile,NLVL is more challenging since it requires align semantic information from both visual and textual modalities.Besides,how to obtain the final timestamp from the alignment relationship is also a tough task.This paper first proposes the pipeline of NLVL,and then categorizes them into supervised and weakly-supervised methods according to whether there is supervised information,following by the analysis of the strengths and weaknesses of each kind of method.Subsequently,the dataset,evaluation protocols and the general performance analysis are presented.Finally,the possible perspectives are obtained by summarizing the existing methods.

Keywords

• multimodal retrieval|video moment localization|video comprehension|cross-modal alignment|cross-modal interaction