Assistive Magnetic Skin System for Speech Reconstruction: An Empowering Technology for Aphonic Individuals

Abstract

Individuals with a voice disorder or speech sound disorder (SSD) (i.e., aphonic or mute people) can have any combination of difficulties with perception, articulation/motor production, and phonotactics, which may impact their speech intelligibility and acceptability, thus finding challenging to communicate with the public. As a result, many individuals suffer from frustration, isolation, and depression. Natural verbal communication for SSD patients is now more feasible than ever thanks to advancements in wearable artificial skins and machine learning. This article presents an assistive magnetic skin system for speech reconstruction (AM2S‐SR), which enables SSD‐afflicted people to communicate with their mouths. Using magnetic field sensors integrated into Magnetphones, the system reads the movement of the mouth by tracking the movement of magnetic skin patches attached next to the bottom lip. The measured magnetic field signals data are then processed using a Fine K‐Nearest Neighbor machine learning classifier. The classified data are then exported verbally on speakers, or visually on a display. AM2S‐SR successfully identifies the full English alphabets with average success rate of 94.96%, thus enabling SSD people to talk using the mouth and have a more natural conversation with others.

Keywords

• E-skin
• machine learning
• magnetic skin
• speech sound diseases
• wearable electronics