A Single Chip SPAD Based Vision Sensing System With Integrated Memristive Spiking Neuromorphic Processing

Abstract

This paper presents a new scalable $8\times8$ single photon avalanche diode (SPAD) based vision sensor with integrated spiking neuromorphic system on a single chip. The proposed vision sensing system adopts the benefits of SPAD’s high quantum efficiency and energy efficiency of memristive spiking neuromorphic processing. The SPAD based vision sensor includes biologically inspired address event representation (AER) readout to generate asynchronous digital address events at the output reducing computation and making it suitable to process directly with integrated on-chip spiking neuromorphic system in a faster and more energy efficient way. A novel on-chip interface is designed to convert the output events of a SPAD-based event sensor into temporally coded spikes (TCS) that enable on-chip processing with integrated spiking neuromorphic system. We have tested the prototype vision sensing system for imaging characters by SPAD based vision sensor and recognizing them using the integrated memristive spiking neuromorphic system. To help with the evaluation, we have built a complete temporal pulses data set from simulating the SPAD vision sensor with AER readout in imaging characters and applied directly to integrated spiking neuromorphic system via designed novel on-chip interface. The achieved accuracy is 89.54% with a power consumption of $316~\mu \text{W}$ for the memristive neuromorphic processor. The SPAD based vision sensor exhibits array-level dynamic range of 148 dB with a power consumption of 2.8 mW. The designed SPAD-based vision sensing system with an integrated spiking neuromorphic system on a single chip shows great promise for robotics, autonomous vehicles, health, and security applications.

Keywords

• Single-photon avalanche diode
• SPAD
• SPAD imager
• vision sensor
• neuromorphic computing
• memristor