Cross-image Text Reading Method Based on Text Line Matching

Abstract

Reading text with a camera can help the computer understand the text content.However,due to the limited field of view of the camera and the complexity of Chinese text recognition,it is sometimes difficult for the computer to read complete text content from a single text image with the camera.Thus,we define the cross-image text reading task,which aims to read the complete text content of a pair of overlapping text images.For the cross-image text reading task,we propose the cross-image text reading method via text line matching.We first adopt a text detection network to crop text lines.Then,we design the text line matching network with the multi-head self-attention mechanism to predict the matching relationships of text lines.Finally,the editing-based text reading network is proposed to remove overlapping texts and read complete text content.We also construct the cross-image Chinese text reading(CCTR) dataset for training and evaluation.Experiment results on CCTR dataset demonstrate that the proposed method achieves higher reading performance than the pixel-level stitching and recognition methods,which proves the superiority of the proposed method.

Keywords

• cross-image text reading|cross-image chinese text reading dataset|text line matching|editing-based text reading|attention mechanism