Artificial visual‐tactile perception array for enhanced memory and neuromorphic computations
Jiaqi He,
Ruilai Wei,
Shuaipeng Ge,
Wenqiang Wu,
Jianchao Guo,
Juan Tao,
Ru Wang,
Chunfeng Wang,
Caofeng Pan
Affiliations
Jiaqi He
CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China
Ruilai Wei
CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China
Shuaipeng Ge
CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China
Wenqiang Wu
Guangdong Research Center for Interfacial Engineering of Functional Materials, College of Materials Science and Engineering Shenzhen University Shenzhen the People's Republic of China
Jianchao Guo
CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China
Juan Tao
CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China
Ru Wang
CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China
Chunfeng Wang
CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China
Caofeng Pan
CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China
Abstract The emulation of human multisensory functions to construct artificial perception systems is an intriguing challenge for developing humanoid robotics and cross‐modal human–machine interfaces. Inspired by human multisensory signal generation and neuroplasticity‐based signal processing, here, an artificial perceptual neuro array with visual‐tactile sensing, processing, learning, and memory is demonstrated. The neuromorphic bimodal perception array compactly combines an artificial photoelectric synapse network and an integrated mechanoluminescent layer, endowing individual and synergistic plastic modulation of optical and mechanical information, including short‐term memory, long‐term memory, paired pulse facilitation, and “learning‐experience” behavior. Sequential or superimposed visual and tactile stimuli inputs can efficiently simulate the associative learning process of “Pavlov's dog”. The fusion of visual and tactile modulation enables enhanced memory of the stimulation image during the learning process. A machine‐learning algorithm is coupled with an artificial neural network for pattern recognition, achieving a recognition accuracy of 70% for bimodal training, which is higher than that obtained by unimodal training. In addition, the artificial perceptual neuron has a low energy consumption of ∼20 pJ. With its mechanical compliance and simple architecture, the neuromorphic bimodal perception array has promising applications in large‐scale cross‐modal interactions and high‐throughput intelligent perceptions.
