Discovering Implied Serial Order Through Model-Free and Model-Based Learning

• Main Authors: Greg Jensen, Herbert S. Terrace, Vincent P. Ferrera
• Format: Article
• Language: English
• Published: Frontiers Media S.A. 2019-08-01
• Series: Frontiers in Neuroscience
• Subjects: reinforcement learning; model-free learning; model-based learning; cognitive maps; transitive inference
• Online Access: https://www.frontiersin.org/article/10.3389/fnins.2019.00878/full

Humans and animals can learn to order a list of items without relying on explicit spatial or temporal cues. To do so, they appear to make use of transitivity, a property of all ordered sets. Here, we summarize relevant research on the transitive inference (TI) paradigm and its relationship to learning the underlying order of an arbitrary set of items. We compare six computational models of TI performance, three of which are model-free (Q-learning, Value Transfer, and REMERGE) and three of which are model-based (RL-Elo, Sequential Monte Carlo, and Betasort). Our goal is to assess the ability of these models to produce empirically observed features of TI behavior. Model-based approaches perform better under a wider range of scenarios, but no single model explains the full scope of behaviors reported in the TI literature.