A sign language smart phone based on realistic animation

Abstract

Mobile phones are now popular in current society for its portability and intelligence, but excluded from life of the hearing-impaired because sounds are unavailable for them. This paper provides a framework of speech based Chinese Sign Language (CSL) Animation System on Smart Phones, which combines speech recognition and sign language animation to make phones convenient for the hearing-impaired. This paper recognizes the Chinese speech into text first, and then proposes a HMMSVM model to judge interrogative expression from the speech and rewrites the plain text as mark-up text according to the rules of Chinese Sign Language Mark-up Language (CSLML) which provides signs for recording expressive context in the text, the corresponding animation is finally synthesized using key-frame technology. The prototype system based on the framework provided in this paper gives plausible Sign Language animation.