The non-effects of repeated exposure to the Cognitive Reflection Test

We estimate the effects of repeated exposure to the Cognitive Reflection Test (CRT) by examining 14,053 MTurk subjects who took the test up to 25 times. In contrast with inferences drawn from self-reported prior exposure to the CRT, we find that prior exposure usually fails to improve scores. On ave...

Full description

Bibliographic Details
Main Authors: Andrew Meyer, Elizabeth Zhou, Shane Frederick
Format: Article
Language:English
Published: Society for Judgment and Decision Making 2018-05-01
Series:Judgment and Decision Making
Subjects:
Online Access:http://journal.sjdm.org/18/18228a/jdm18228a.pdf
Description
Summary:We estimate the effects of repeated exposure to the Cognitive Reflection Test (CRT) by examining 14,053 MTurk subjects who took the test up to 25 times. In contrast with inferences drawn from self-reported prior exposure to the CRT, we find that prior exposure usually fails to improve scores. On average, respondents get only 0.024 additional items correct per exposure, and this small increase is driven entirely by the minority of subjects who continue to spend time reflecting on the items. Moreover, later scores retain the predictive validity of earlier scores, even when they differ, because initial success and later improvement appear to measure the same thing.
ISSN:1930-2975