Is the treatment weak or the test insensitive: Interrogating item difficulties to elucidate the nature of reading intervention effects

Intervention research in education is sometimes criticized for the use of experimenter developed assessments, especially when these are over aligned with treatment. At the same time, intervention researchers sometimes prefer locally developed assessments because they appear to be more sensitive to t...

Full description

Bibliographic Details
Main Authors: Francis, D.J (Author), Khalaf, S. (Author), Kulesz, P.A (Author), Vaughn, S.R (Author), Walczak, M. (Author)
Format: Article
Language:English
Published: Elsevier Ltd 2022
Online Access:View Fulltext in Publisher
LEADER 01871nam a2200181Ia 4500
001 10.1016-j.lindif.2022.102167
008 220706s2022 CNT 000 0 und d
020 |a 10416080 (ISSN) 
245 1 0 |a Is the treatment weak or the test insensitive: Interrogating item difficulties to elucidate the nature of reading intervention effects 
260 0 |b Elsevier Ltd  |c 2022 
856 |z View Fulltext in Publisher  |u https://doi.org/10.1016/j.lindif.2022.102167 
520 3 |a Intervention research in education is sometimes criticized for the use of experimenter developed assessments, especially when these are over aligned with treatment. At the same time, intervention researchers sometimes prefer locally developed assessments because they appear to be more sensitive to treatment effects even when the test is not subject to the criticism of over alignment. This paper examines the question of test sensitivity to treatment effects for experimenter developed and standardized tests for the specific case of reading in grade 8. We examine similarities and differences between a specific experimenter developed test and widely used standardized reading assessment. Analyses show these particular tests to be quite comparable. The paper concludes with an examination of test sensitivity by simulating treatment effects of different magnitudes. These analyses highlight some potential limitations of the standardized test for detecting small to moderate effects depending on the ability range of the students participating in intervention. The implications for intervention research and identification of students under response to intervention are discussed. © 2022 
700 1 |a Francis, D.J.  |e author 
700 1 |a Khalaf, S.  |e author 
700 1 |a Kulesz, P.A.  |e author 
700 1 |a Vaughn, S.R.  |e author 
700 1 |a Walczak, M.  |e author 
773 |t Learning and Individual Differences