| Summary: | The current study evaluated the implementation of evidence-based reading interventions using a multifaceted implementation measurement approach. Multilevel modeling was used to examine how three direct measures of implementation related to each other and to student academic outcomes and to examine patterns of implementation across time. Eight instructional groups were video taped weekly for nine weeks, and pre- and post-test assessments were given to 31 at-risk kindergartners from two schools using established evidence-based practices. Each implementation measure represented a different measurement approach (i.e., discrete behavioral measurement, global ratings) and focused on different aspects of implementation (e.g., structural, process, or multicomponent elements). Overall, results of this analysis indicated that (a) the implementation tools were highly correlated with each other, (b) only the multicomponent tool independently accounted for group differences, (c) together the multicomponent and process-oriented measures appear to account for additional variance in group differences, and (d) there were no significant trends in implementation across time as measured by any of the tools, however there were significant differences in trends over time between groups when using the structural measure. Implications for research and practice are discussed, including the importance of taking a multifaceted approach to measuring implementation and aligning implementation measures with program theory.
|
|---|
