Vibrotactile Stimulation for Emotional Elicitation During Audiovisual Events

Abstract

Audiovisual media are an essential source of information and leisure in our lives, but people with hearing disabilities lose the emotional content that is intended to be transmitted through this channel. Some previous studies have tried to find alternative ways to elicit these emotions using other sensory channels, and some attempts have been made with haptic systems synchronized with rhythms of music and action in movies, but they fail in studying deeply the provoked neural response. In this article, we propose the tactile channel as the alternative to the auditory channel to convey the emotions generated by audiovisual events, measuring the effect of a proprietary system in the emotions by monitoring neural response. We have developed a low-cost system consisting of an Arduino Uno, a pair of gloves with micromotors and a driver. Using EEG, we have measured brain activity generated during audiovisual stimulation of a short film with clearly negative emotional content, with and without synchronized tactile stimuli, in two groups of people, one without hearing impairment and the other with hearing impairment. We have used the Hotelling’s T2 test as a multivariate statistical analysis of EEG measurements. Comparing the difference in brain activation generated in the conditions with and without vibration in both groups, we have found that multimodal stimulation by touch and associated with audiovisual stimuli enhances in a 30% the emotional, attentional, and auditory circuits activation, due to brain reorganization around frontal and parietal structures (all related to emotional responses) in the hearing-impaired group.

Keywords

• Accessibility
• audiovisual
• EEG
• emotions
• hearing impairment
• multisensorial