Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies
Behaviour in spatial navigation is often organised into map-based (place-driven) versus map-free (cue-driven) strategies; behaviour in operant conditioning research is often organised into goal-directed versus habitual strategies. Here we attempt to unify the two. We review one powerful theory for d...
Main Authors: | , |
---|---|
Format: | Article |
Language: | English |
Published: |
Frontiers Media S.A.
2012-11-01
|
Series: | Frontiers in Behavioral Neuroscience |
Subjects: | |
Online Access: | http://journal.frontiersin.org/Journal/10.3389/fnbeh.2012.00079/full |
id |
doaj-fda7e69caf2945f8b88773db560eac39 |
---|---|
record_format |
Article |
spelling |
doaj-fda7e69caf2945f8b88773db560eac392020-11-24T22:57:22ZengFrontiers Media S.A.Frontiers in Behavioral Neuroscience1662-51532012-11-01610.3389/fnbeh.2012.0007930039Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategiesMehdi eKhamassi0Mehdi eKhamassi1Mark D Humphries2Mark D Humphries3Mark D Humphries4Université Pierre et Marie Curie - Paris 6CNRS (Centre National de la Recherche Scientifique)Ecole Normale SuperieureUniversity of SheffieldUniversity of ManchesterBehaviour in spatial navigation is often organised into map-based (place-driven) versus map-free (cue-driven) strategies; behaviour in operant conditioning research is often organised into goal-directed versus habitual strategies. Here we attempt to unify the two. We review one powerful theory for distinct forms of learning during instrumental conditioning, namely model-based (maintaining a representation of the world) and model-free (reacting to immediate stimuli) learning algorithms. We extend these lines of argument to propose an alternative taxonomy for spatial navigation, showing how various previously identified strategies can be distinguished as model-based or model-free depending on the usage of information and not on the type of information (e.g. cue vs place). We argue that identifying model-free learning with dorsolateral striatum and model-based learning with dorsomedial striatum could reconcile numerous conflicting results in the spatial navigation literature. From this perspective, we further propose that the ventral striatum plays key roles in the model-building process. We propose that the core of the ventral striatum is positioned to learn the probability of action selection for every transition between states of the world. We further review suggestions that the ventral striatal core and shell are positioned to act as critics contributing to the computation of a reward prediction error for model-free and model-based systems, respectively.http://journal.frontiersin.org/Journal/10.3389/fnbeh.2012.00079/fullBasal GangliaNucleus Accumbensreinforcement learningaction-outcomestimulus-response |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Mehdi eKhamassi Mehdi eKhamassi Mark D Humphries Mark D Humphries Mark D Humphries |
spellingShingle |
Mehdi eKhamassi Mehdi eKhamassi Mark D Humphries Mark D Humphries Mark D Humphries Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies Frontiers in Behavioral Neuroscience Basal Ganglia Nucleus Accumbens reinforcement learning action-outcome stimulus-response |
author_facet |
Mehdi eKhamassi Mehdi eKhamassi Mark D Humphries Mark D Humphries Mark D Humphries |
author_sort |
Mehdi eKhamassi |
title |
Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies |
title_short |
Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies |
title_full |
Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies |
title_fullStr |
Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies |
title_full_unstemmed |
Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies |
title_sort |
integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies |
publisher |
Frontiers Media S.A. |
series |
Frontiers in Behavioral Neuroscience |
issn |
1662-5153 |
publishDate |
2012-11-01 |
description |
Behaviour in spatial navigation is often organised into map-based (place-driven) versus map-free (cue-driven) strategies; behaviour in operant conditioning research is often organised into goal-directed versus habitual strategies. Here we attempt to unify the two. We review one powerful theory for distinct forms of learning during instrumental conditioning, namely model-based (maintaining a representation of the world) and model-free (reacting to immediate stimuli) learning algorithms. We extend these lines of argument to propose an alternative taxonomy for spatial navigation, showing how various previously identified strategies can be distinguished as model-based or model-free depending on the usage of information and not on the type of information (e.g. cue vs place). We argue that identifying model-free learning with dorsolateral striatum and model-based learning with dorsomedial striatum could reconcile numerous conflicting results in the spatial navigation literature. From this perspective, we further propose that the ventral striatum plays key roles in the model-building process. We propose that the core of the ventral striatum is positioned to learn the probability of action selection for every transition between states of the world. We further review suggestions that the ventral striatal core and shell are positioned to act as critics contributing to the computation of a reward prediction error for model-free and model-based systems, respectively. |
topic |
Basal Ganglia Nucleus Accumbens reinforcement learning action-outcome stimulus-response |
url |
http://journal.frontiersin.org/Journal/10.3389/fnbeh.2012.00079/full |
work_keys_str_mv |
AT mehdiekhamassi integratingcorticolimbicbasalgangliaarchitecturesforlearningmodelbasedandmodelfreenavigationstrategies AT mehdiekhamassi integratingcorticolimbicbasalgangliaarchitecturesforlearningmodelbasedandmodelfreenavigationstrategies AT markdhumphries integratingcorticolimbicbasalgangliaarchitecturesforlearningmodelbasedandmodelfreenavigationstrategies AT markdhumphries integratingcorticolimbicbasalgangliaarchitecturesforlearningmodelbasedandmodelfreenavigationstrategies AT markdhumphries integratingcorticolimbicbasalgangliaarchitecturesforlearningmodelbasedandmodelfreenavigationstrategies |
_version_ |
1725651045466505216 |