Unsupervised Conditional Reflex Learning Based on Convolutional Spiking Neural Network and Reward Modulation

Abstract

Automatic decision and lane-keeping tasks are the most fundamental but important tasks in robot controlling researches. Concerned about the computing limitations of mobile robot platforms, an easily trainable method with low computational consumption and low latency is needed to solve this problem. An unsupervised conditional reflex learning network was proposed in this paper, which uses the conditional pattern to learn the conditional pattern and make decisions in an unsupervised manner. We used the convolutional spiking neural network to extract hidden features of road lanes, and then used the dopamine modulation mechanism to learn the decision-making information from the acquired features. In order to evaluate the quality of automatic decision-making models, two metrics were designed which are total deviation distance per second (TDDPS) and target achievement rate during training (TAR). In the process of training, neither labels were given to the convolutional spiking neural network, nor artificial decision-making information was assigned to the dopamine modulation layer. Simulation experiments showed that the proposed model has a state-of-the-art performance in a relatively complicated scenario and solves three limitations mentioned in previous works. Our work brought more biological inspiration into decision support systems, with the hope that the proposed method can promote the development of the bionic decision support system in hardware, especially in neuromorphic hardware.

Keywords

• Decision support systems
• dopamine modulation
• reinforcement learning
• unsupervised learning
• spiking neural networks