Oral Retelling as a Measure of Reading Comprehension: The Generalizability of Ratings of Elementary School Students Reading Expository Texts

• Main Author: Burton, Rachel Clinger
• Format: Others
• Published: BYU ScholarsArchive 2008
• Subjects: oral retelling; reading comprehension; elementary students; expository texts; generalizability theory; Communication Sciences and Disorders
• Online Access: https://scholarsarchive.byu.edu/etd/1678; https://scholarsarchive.byu.edu/cgi/viewcontent.cgi?article=2677&context=etd

The purpose of this study was to refine a rating procedure used to assess intermediate elementary school students' ability to orally retell what they had read from two expository passages. Oral retellings from 28 fourth grade students were tape-recorded and rated on two different occasions by each of 4 raters. A four-facet (passage, day of test administration, rater, and rating occasion) generalizability study was conducted using a partially nested design. The six largest sources of variability identified in the G-study included (a) students, (b) the student-by-day interaction, (c) the interaction of passage with rater (nested within student and day), (d) the student-by-day-by-occasion interaction, (e) the passage-by-raters (nested within students and day)-by-occasion interaction, and (f) the residual. A D-study was conducted to predict the values of the error variances and generalizability indices for both relative and absolute decisions. The results show how the error variance and the generalizability coefficients vary as a function of the number of passages, days of test administration, raters, and rating occasions. The results of the D study indicate that adding an extra reading day would produce a greater increase in reliability than asking the students to read more passages, or using more raters or more rating occasions. To achieve the greatest gain in generalizability, teachers should have students read at least two passages on at least two separate days and have their retelling rated by at least two raters and then compute a mean rating for each student averaged across the various passages, testing days, and raters.