Binary ROCs and Their Implications for the Measurement of Memory

• Main Author: Dube, Chad
• Format: Others
• Published: ScholarWorks@UMass Amherst 2011
• Subjects: Psychology
• Online Access: https://scholarworks.umass.edu/open_access_dissertations/439; https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1445&context=open_access_dissertations

Bröder and Schütz (2009) have argued that the curvature typically observed in recognition memory receiver-operating characteristics (ROCs) is a by-product of the ratings task often used to obtain them. According to those authors, ROCs collected by experimentally manipulating response bias are linear and consistent with the assumptions of threshold and multinomial processing tree (MPT) models. Two experiments are reported which are broadly consistent with previous work by Dube and Rotello (under review) in showing that ROCs are curved and consistent with signal detection theory (SDT) regardless of the procedure used to obtain them. These results have implications for how accuracy is measured in tasks requiring binary responses. It is suggested that the use of statistics consistent with the assumptions of threshold models (e.g. percent correct, hits minus false alarms) should be avoided, as they are likely to produce statistical errors in several areas of the literature where ROCs have been found to be curved (Rotello, Masson, & Verde, 2008; Dube, Rotello, & Heit, 2010; 2011). SDT-based measures and ROC analysis are recommended to complement or replace analyses based on threshold statistics.