Artificial Neural Network Learning, Attention, and Memory

Abstract

The learning equations of an ANN are presented, giving an extremely concise derivation based on the principle of backpropagation through the descendent gradient. Then, a dual network is outlined acting between synapses of a basic ANN, which controls the learning process and coordinates the subnetworks selected by attention mechanisms toward purposeful behaviors. Mechanisms of memory and their affinity with comprehension are considered, by emphasizing the common role of abstraction and the interplay between assimilation and accommodation, in the spirit of Piaget’s analysis of psychological acquisition and genetic epistemology. Learning, comprehension, and knowledge are expressed as different levels of organization of informational processes inside cognitive systems. It is argued that formal analyses of cognitive artificial systems could shed new light on typical mechanisms of “natural intelligence” and, in a specular way, that models of natural cognition processes could promote further developments of ANN models. Finally, new possibilities of chatbot interaction are briefly discussed.

Keywords

• artificial neural networks
• Machine Learning
• artificial intelligence
• cognitive systems
• back-propagation