Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies

Abstract

Read online

Behaviour in spatial navigation is often organised into map-based (place-driven) versus map-free (cue-driven) strategies; behaviour in operant conditioning research is often organised into goal-directed versus habitual strategies. Here we attempt to unify the two. We review one powerful theory for distinct forms of learning during instrumental conditioning, namely model-based (maintaining a representation of the world) and model-free (reacting to immediate stimuli) learning algorithms. We extend these lines of argument to propose an alternative taxonomy for spatial navigation, showing how various previously identified strategies can be distinguished as model-based or model-free depending on the usage of information and not on the type of information (e.g. cue vs place). We argue that identifying model-free learning with dorsolateral striatum and model-based learning with dorsomedial striatum could reconcile numerous conflicting results in the spatial navigation literature. From this perspective, we further propose that the ventral striatum plays key roles in the model-building process. We propose that the core of the ventral striatum is positioned to learn the probability of action selection for every transition between states of the world. We further review suggestions that the ventral striatal core and shell are positioned to act as critics contributing to the computation of a reward prediction error for model-free and model-based systems, respectively.

Keywords

• Basal Ganglia
• Nucleus Accumbens
• reinforcement learning
• action-outcome
• stimulus-response