Reconstructing Multi-Stroke Characters From Brain Signals Toward Generalizable Handwriting Brain–Computer Interfaces

Abstract

Handwriting Brain-Computer Interfaces (BCIs) provides a promising communication avenue for individuals with paralysis. While English-based handwriting BCIs have achieved rapid typewriting with 26 lowercase letters (mostly containing one stroke each), it is difficult to extend to complex characters, especially those with multiple strokes and large character sets. The Chinese characters, including over 3500 commonly used characters with 10.3 strokes per character on average, represent a highly complex writing system. This paper proposes a Chinese handwriting BCI system, which reconstructs multi-stroke handwriting trajectories from brain signals. Through the recording of cortical neural signals from the motor cortex, we reveal distinct neural representations for stroke-writing and pen-lift phases. Leveraging this finding, we propose a stroke-aware approach to decode stroke-writing trajectories and pen-lift movements individually, which can reconstruct recognizable characters (accuracy of 86% with 400 characters). Our approach demonstrates high stability over 5 months, shedding light on generalized and adaptable handwriting BCIs.

Keywords

• Brain-computer interfaces (BCIs)
• neural signal decoding
• handwriting reconstruction
• multi-stroke characters