Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies

Behaviour in spatial navigation is often organised into map-based (place-driven) versus map-free (cue-driven) strategies; behaviour in operant conditioning research is often organised into goal-directed versus habitual strategies. Here we attempt to unify the two. We review one powerful theory for d...

Full description

Bibliographic Details
Main Authors: Mehdi eKhamassi, Mark D Humphries
Format: Article
Language:English
Published: Frontiers Media S.A. 2012-11-01
Series:Frontiers in Behavioral Neuroscience
Subjects:
Online Access:http://journal.frontiersin.org/Journal/10.3389/fnbeh.2012.00079/full
id doaj-fda7e69caf2945f8b88773db560eac39
record_format Article
spelling doaj-fda7e69caf2945f8b88773db560eac392020-11-24T22:57:22ZengFrontiers Media S.A.Frontiers in Behavioral Neuroscience1662-51532012-11-01610.3389/fnbeh.2012.0007930039Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategiesMehdi eKhamassi0Mehdi eKhamassi1Mark D Humphries2Mark D Humphries3Mark D Humphries4Université Pierre et Marie Curie - Paris 6CNRS (Centre National de la Recherche Scientifique)Ecole Normale SuperieureUniversity of SheffieldUniversity of ManchesterBehaviour in spatial navigation is often organised into map-based (place-driven) versus map-free (cue-driven) strategies; behaviour in operant conditioning research is often organised into goal-directed versus habitual strategies. Here we attempt to unify the two. We review one powerful theory for distinct forms of learning during instrumental conditioning, namely model-based (maintaining a representation of the world) and model-free (reacting to immediate stimuli) learning algorithms. We extend these lines of argument to propose an alternative taxonomy for spatial navigation, showing how various previously identified strategies can be distinguished as model-based or model-free depending on the usage of information and not on the type of information (e.g. cue vs place). We argue that identifying model-free learning with dorsolateral striatum and model-based learning with dorsomedial striatum could reconcile numerous conflicting results in the spatial navigation literature. From this perspective, we further propose that the ventral striatum plays key roles in the model-building process. We propose that the core of the ventral striatum is positioned to learn the probability of action selection for every transition between states of the world. We further review suggestions that the ventral striatal core and shell are positioned to act as critics contributing to the computation of a reward prediction error for model-free and model-based systems, respectively.http://journal.frontiersin.org/Journal/10.3389/fnbeh.2012.00079/fullBasal GangliaNucleus Accumbensreinforcement learningaction-outcomestimulus-response
collection DOAJ
language English
format Article
sources DOAJ
author Mehdi eKhamassi
Mehdi eKhamassi
Mark D Humphries
Mark D Humphries
Mark D Humphries
spellingShingle Mehdi eKhamassi
Mehdi eKhamassi
Mark D Humphries
Mark D Humphries
Mark D Humphries
Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies
Frontiers in Behavioral Neuroscience
Basal Ganglia
Nucleus Accumbens
reinforcement learning
action-outcome
stimulus-response
author_facet Mehdi eKhamassi
Mehdi eKhamassi
Mark D Humphries
Mark D Humphries
Mark D Humphries
author_sort Mehdi eKhamassi
title Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies
title_short Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies
title_full Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies
title_fullStr Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies
title_full_unstemmed Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies
title_sort integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies
publisher Frontiers Media S.A.
series Frontiers in Behavioral Neuroscience
issn 1662-5153
publishDate 2012-11-01
description Behaviour in spatial navigation is often organised into map-based (place-driven) versus map-free (cue-driven) strategies; behaviour in operant conditioning research is often organised into goal-directed versus habitual strategies. Here we attempt to unify the two. We review one powerful theory for distinct forms of learning during instrumental conditioning, namely model-based (maintaining a representation of the world) and model-free (reacting to immediate stimuli) learning algorithms. We extend these lines of argument to propose an alternative taxonomy for spatial navigation, showing how various previously identified strategies can be distinguished as model-based or model-free depending on the usage of information and not on the type of information (e.g. cue vs place). We argue that identifying model-free learning with dorsolateral striatum and model-based learning with dorsomedial striatum could reconcile numerous conflicting results in the spatial navigation literature. From this perspective, we further propose that the ventral striatum plays key roles in the model-building process. We propose that the core of the ventral striatum is positioned to learn the probability of action selection for every transition between states of the world. We further review suggestions that the ventral striatal core and shell are positioned to act as critics contributing to the computation of a reward prediction error for model-free and model-based systems, respectively.
topic Basal Ganglia
Nucleus Accumbens
reinforcement learning
action-outcome
stimulus-response
url http://journal.frontiersin.org/Journal/10.3389/fnbeh.2012.00079/full
work_keys_str_mv AT mehdiekhamassi integratingcorticolimbicbasalgangliaarchitecturesforlearningmodelbasedandmodelfreenavigationstrategies
AT mehdiekhamassi integratingcorticolimbicbasalgangliaarchitecturesforlearningmodelbasedandmodelfreenavigationstrategies
AT markdhumphries integratingcorticolimbicbasalgangliaarchitecturesforlearningmodelbasedandmodelfreenavigationstrategies
AT markdhumphries integratingcorticolimbicbasalgangliaarchitecturesforlearningmodelbasedandmodelfreenavigationstrategies
AT markdhumphries integratingcorticolimbicbasalgangliaarchitecturesforlearningmodelbasedandmodelfreenavigationstrategies
_version_ 1725651045466505216