The Integration of Automated Essay Scoring Systems into the Equating Process for Mixed-Format Tests

• Other Authors: Olgar, Süleyman (authoraut)
• Format: Others
• Language: English; English
• Published: Florida State University
• Subjects: Educational tests and measurements; Educational psychology; Educational technology
• Online Access: http://purl.flvc.org/fsu/fd/FSU_migr_etd-9661

With the latest developments in computer based testing, implementing equating techniques that incorporate automated essay scoring systems such as e-rater are encouraging potential new directions for equating mixed-format tests of writing that include multiple-choice (MC) items and an essay. The inclusion of generic e-rater essay scores into the anchor set was used to adjust for essay prompts' difficulty. A pseudo-test form study was completed to investigate the impact of using generic e-rater scores to equate mixed-format tests that consist of MC items and a single essay. Equating outcomes for the proposed equating methodology (MC+e-rater scores) as an alternative to the current MC-only anchor-item set approach for the mixed-format tests revealed some promising results. The kappa and observed agreements for pass/fail status determined from the composite scores were very large and similar for all six equating methods comparing the alternate method to the current approach. The findings indicate that MC+e-rater equating outcomes are as strong as the MC-only equating results, and even better for some conditions. === A Dissertation submitted to the Department of Educational Psychology and Learning Systems in partial fulfillment of the requirements for the degree of Doctor of Philosophy. === Summer Semester 2015. === May 28, 2015. === automated essay scoring, certification tests, Equating, mixed format tests, pass/fail agreements, pseudo test forms === Includes bibliographical references. === Russell George Almond, Professor Directing Dissertation; Elizabeth Jakubowski, University Representative; Betsy Jane Becker, Committee Member; Insu Paek, Committee Member.