Development of an Online Home Appliance Control System Using Augmented Reality and an SSVEP-Based Brain–Computer Interface

Abstract

In this study, we implemented a new home appliance control system by combining electroencephalography (EEG)-based brain-computer interface (BCI), augmented reality (AR), and internet of things (IoT) technologies. We adopted a steady-state visual evoked potential (SSVEP)-based BCI paradigm for the implementation of a fast and robust BCI system. In the offline experiment, we compared the performances of three BCIs adopting different types of visual stimuli in an AR environment to determine the optimal visual stimulus. In the online experiment, we evaluated the feasibility of the proposed smart home system using the optimal stimulus by controlling three home appliances in real time. The visual stimuli were presented on a see-through head-mounted display (HMD), while the recorded brain activity was analyzed to classify the control command, and the home appliances were controlled through IoT. In the offline experiment, a grow/shrink stimulus (GSS) consisting of a star-shaped flickering object of varying size was selected as the optimal stimulus, eliciting SSVEP responses more effectively than the other options. In the online experiment, all users could turn the BCI-based control system on/off whenever they wanted using the eye-blinking-based electrooculogram (EOG) switch, and could successfully perform all the designated control tasks without difficulty. The average classification accuracy of the SSVEP-BCI-based control system was 92.8%, with an information transfer rate (ITR) of 37.4 bits/min. The proposed system exhibited an excellent performance, surpassing the best results reported in previous studies regarding external device control based on BCI using an HMD as rendering device.

Keywords

• Augmented reality
• brain-computer interface
• electroencephalography
• internet of things
• steady-state visual evoked potential