Time Information Integration Network for Event Cameras

Abstract

Event cameras are asynchronous sensors that operate in a completely different way from traditional cameras.Rather than catching pictures at a steady rate,event cameras measure light changes (called events) separately for every pixel.As a sequence,it alleviates the problems of traditional cameras in complex light conditions and scenes where objects move at high speed.With the development of convolutional neural networks,learning-based pattern recognition methods have made great progress in visual tasks such as optical flow estimation and target recognition by converting the output of the event camera into a pseudo-ima-ge representation.However,such methods abandon the temporal correlation between the event streams,so that the texture of the pseudo image is not clear enough,and it is difficult to extract the features.The key to solving this problem lies in how to model relevant information between events in the sample.Therefore,a neural network framework based on event stream partition algorithm is proposed,which explicitly integrates the temporal information of event streams.The framework divides the incoming stream of events into several parts,and a weight distribution network assigns different weights to each piece of the streams.Then,the framework uses convolutional neural network to fuse temporal information and extract advanced features.Finally,the input sample is classified.We thoroughly validate the proposed framework on object recognition.Comparison experiments on N-Caltech101 and N-cars datasets show that the proposed framework has a significant improvement in classification accuracy compared with the most advanced existing algorithms.

Keywords

• temporal information|event streams|fusion|weight allocation|convolutional neural network