Combining Accuracy and Plasticity in Convolutional Neural Networks Based on Resistive Memory Arrays for Autonomous Learning

Abstract

Nowadays, artificial neural networks (ANNs) can outperform the human brain’s ability in specific tasks. However, ANNs cannot replicate the efficient and low-power learning, adaptation, and consolidation typical of biological organisms. Here, we present a hardware design based on arrays of SiOx resistive switching random-access memory (RRAM), which allows combining the accuracy of convolutional neural networks with the flexibility of bio-inspired neuronal plasticity. In order to enable the combination of the stable and the plastic attributes of the network, we exploit the spike-frequency adaptation of the neurons relying on the multilevel programming of the RRAM devices. This procedure enhances the efficiency and accuracy of the network for MNIST, noisy MNIST (N-MNIST), Fashion-MNIST, and CIFAR-10 datasets, with inference accuracies of about 99%–89%, respectively. We also demonstrate that the hardware is capable of asynchronous self-adaptation of its operative frequency according to the fire rate of the spiking neuron, thus optimizing the whole behavior of the network. We finally show that the system enables fast and accurate filter retraining to overcome catastrophic forgetting, showing high efficiency in terms of operations per second and robustness against device non-idealities. This work paves the way for the theoretical modeling and hardware realization of resilient autonomous systems in dynamic environments.

Keywords

• Catastrophic forgetting
• complementary learning systems
• continual learning
• convolutional neural networks (CNNs)
• resistive switching random-access memory (RRAM)
• spike-timing-dependent plasticity (STDP)