Tactile Audio Responsive Intelligent System

Abstract

For people with visual impairments, information encoded in a visual format creates certain barriers. To alleviate this, a large volume of research has been conducted in the field of assistive technology. In our work, we developed a special system that makes educational materials more accessible. The system consists of three components: the pre-labelled tactile graphics, an interactive labelling web tool and the phone application. Tactile graphics are used at schools for the blind and allow the students to understand non-textual information by touch. The digital version of the graphics first needs to be labelled by teachers using the developed web tool. Then, the phone app, which is based on the Android platform, will accompany those graphics with the audio descriptions. The fundamental purpose of the developed app is to allow the user to gain information without sighted assistance. We also conducted a study to evaluate the system. First, the structured interview was carried out to gather data about the participant’s experience with the tactile graphics and mobile devices. Next, quantitative measurements were obtained through a series of experiments. Subsequently, a post-experimental session was carried out to record the participants’ thoughts and opinions about the system. The results of the experiments demonstrated that the proposed mobile application allows the users to explore the graphics more efficiently.

Keywords

• Assistive technology
• audio feedback
• computer vision
• fingertip detection
• tactile graphics
• visually impaired