Survey of Sign Language Recognition and Translation

Abstract

Different from spoken languages, sign language is mainly composed of continuous gestures. Sign langu-age recognition and translation are important means of facilitating barrier-free communication between the hearing-impaired and the hearing person. The sign language recognition and translation research task is a typical multi-domain cross-study by processing and analyzing sign language videos and displaying the recognition results in text form. In recent years, sign language recognition and translation research based on deep learning has made great progress. In order to facilitate researchers to systematically and comprehensively understand the research tasks of sign language recognition and translation, the review work is carried out from the perspectives of sign language recognition and sign language translation. Firstly, the translation research work is classified and summarized and its characteristics are analyzed. Secondly, the common sign language recognition and translation research datasets of different countries are summarized and classified from the perspectives of isolated sign language words and continuous sign language sentences. Based on the difference in research tasks, the corresponding evaluation index system is introduced. Finally, the major challenges of current research on sign language recognition and translation are summarized from the aspects of effective information extraction of sign language visual features, multi-cue weight assignment, relationship between sign language and natural language grammar, and sign language dataset resources.

Keywords

• |sign language recognition|sign language translation|deep learning|neural network