Program Evaluation: Effective Behavioral and Instructional Support Systems and Student Reading Outcomes

This program evaluation studied Effective Behavioral and Instructional Support Systems (EBISS), a Response to Intervention (RTI) initiative focused at changing district leadership behaviors to close the policy-research-practice gap and improve students' reading outcomes at third grade. A pre/po...

Full description

Bibliographic Details
Main Author: Ryan Jackson, Kathleen
Other Authors: Kamata, Akihito
Language:en_US
Published: University of Oregon 2014
Subjects:
Online Access:http://hdl.handle.net/1794/17927
id ndltd-uoregon.edu-oai-scholarsbank.uoregon.edu-1794-17927
record_format oai_dc
spelling ndltd-uoregon.edu-oai-scholarsbank.uoregon.edu-1794-179272019-02-02T17:24:29Z Program Evaluation: Effective Behavioral and Instructional Support Systems and Student Reading Outcomes Ryan Jackson, Kathleen Kamata, Akihito Early literacy Leadership Policy This program evaluation studied Effective Behavioral and Instructional Support Systems (EBISS), a Response to Intervention (RTI) initiative focused at changing district leadership behaviors to close the policy-research-practice gap and improve students' reading outcomes at third grade. A pre/post quasi-experimental comparison using a matched group design evaluated the four-year initiative. EBISS districts (n = 25) were matched to non-EBISS districts n = 25) by important district variables of: (a) size (small, medium, and large) and (b) region (coast, central, east, south, and valley), (c) students receiving Free and Reduced Meals (FARM), (d) students who were white and non-white, (e) students who were male and female, (f) years of teacher's experience, and (g) third grade Oregon Assessment of Knowledge and Skills (OAKS) scores. The aim of this research was to answer two questions. The first asked whether the performance of students in non-EBISS and EBISS districts were significantly different when measured by their percent of students passing the third grade reading OAKS in 2006-07 (EBISS pre-treatment year) and 2010-11 (EBISS post-treatment year). The second question analyzed whether EBISS districts with high implementation scores made statistically significantly different gains from EBISS districts with low implementation scores on their percent of students who passed the third grade OAKS reading assessment from 2006-07 to 2012-11. The results of the first research question indicated no significant differences for group (p = .312) or time (p = .488) between EBISS and non-EBISS districts scores on the OAKS reading test at third grade. Similarly, the second research question results indicated no significant differences (p = .452) between EBISS districts with high and low implementation scores on the OAKS reading scores at third grade. Findings are discussed in relation to: (a) the challenge when measuring administrator effectiveness based on distal outcomes, (b) confounding variables that affected internal and external validity, and (c) how this study informs the future design of evaluation research, in the pre-implementation year of an initiative, so variables that are know to be effective in improving student outcomes can be replicated. 2014-06-17T19:45:02Z 2014-06-17T19:45:02Z 2014-06-17 Electronic Thesis or Dissertation http://hdl.handle.net/1794/17927 en_US All Rights Reserved. University of Oregon
collection NDLTD
language en_US
sources NDLTD
topic Early literacy
Leadership
Policy
spellingShingle Early literacy
Leadership
Policy
Ryan Jackson, Kathleen
Program Evaluation: Effective Behavioral and Instructional Support Systems and Student Reading Outcomes
description This program evaluation studied Effective Behavioral and Instructional Support Systems (EBISS), a Response to Intervention (RTI) initiative focused at changing district leadership behaviors to close the policy-research-practice gap and improve students' reading outcomes at third grade. A pre/post quasi-experimental comparison using a matched group design evaluated the four-year initiative. EBISS districts (n = 25) were matched to non-EBISS districts n = 25) by important district variables of: (a) size (small, medium, and large) and (b) region (coast, central, east, south, and valley), (c) students receiving Free and Reduced Meals (FARM), (d) students who were white and non-white, (e) students who were male and female, (f) years of teacher's experience, and (g) third grade Oregon Assessment of Knowledge and Skills (OAKS) scores. The aim of this research was to answer two questions. The first asked whether the performance of students in non-EBISS and EBISS districts were significantly different when measured by their percent of students passing the third grade reading OAKS in 2006-07 (EBISS pre-treatment year) and 2010-11 (EBISS post-treatment year). The second question analyzed whether EBISS districts with high implementation scores made statistically significantly different gains from EBISS districts with low implementation scores on their percent of students who passed the third grade OAKS reading assessment from 2006-07 to 2012-11. The results of the first research question indicated no significant differences for group (p = .312) or time (p = .488) between EBISS and non-EBISS districts scores on the OAKS reading test at third grade. Similarly, the second research question results indicated no significant differences (p = .452) between EBISS districts with high and low implementation scores on the OAKS reading scores at third grade. Findings are discussed in relation to: (a) the challenge when measuring administrator effectiveness based on distal outcomes, (b) confounding variables that affected internal and external validity, and (c) how this study informs the future design of evaluation research, in the pre-implementation year of an initiative, so variables that are know to be effective in improving student outcomes can be replicated.
author2 Kamata, Akihito
author_facet Kamata, Akihito
Ryan Jackson, Kathleen
author Ryan Jackson, Kathleen
author_sort Ryan Jackson, Kathleen
title Program Evaluation: Effective Behavioral and Instructional Support Systems and Student Reading Outcomes
title_short Program Evaluation: Effective Behavioral and Instructional Support Systems and Student Reading Outcomes
title_full Program Evaluation: Effective Behavioral and Instructional Support Systems and Student Reading Outcomes
title_fullStr Program Evaluation: Effective Behavioral and Instructional Support Systems and Student Reading Outcomes
title_full_unstemmed Program Evaluation: Effective Behavioral and Instructional Support Systems and Student Reading Outcomes
title_sort program evaluation: effective behavioral and instructional support systems and student reading outcomes
publisher University of Oregon
publishDate 2014
url http://hdl.handle.net/1794/17927
work_keys_str_mv AT ryanjacksonkathleen programevaluationeffectivebehavioralandinstructionalsupportsystemsandstudentreadingoutcomes
_version_ 1718970147221274624