Validating the Performance Standards for Cut Scores of 2006 Taiwan Assessment of Student Achievement in Mathematics

• Main Authors: Yi-Fang Wu, 吳宜芳
• Other Authors: Huey-Ing Tzou, Chuan-Ju Lin
• Format: Others
• Language: zh-TW
• Published: 2007
• Online Access: http://ndltd.ncl.edu.tw/handle/19224867713212201006

碩士 === 國立臺南大學 === 測驗統計研究所 === 95 === The purpose of the study was to provide the validation of the performance standards for cut scores of 2006 Taiwan Assessment of Student Achievement in Mathematics (TASA-MA). In order to testify and examine the improvement of consistency, the study applied different procedures and approaches to the modified Angoff standard setting method, including feedbacks with empirical difficulties and Reckase charts, and test items with/without item-grouping in advance. A generalizability study was performed to investigate the sources and magnitude of the variation of results in terms of several facets-- item, panelist, and item difficulty. The data came from two sets of items in TASA-MA, and one was taken by 4th graders and the other was for 6th graders. Each of the test sets consists of 104 multiple-choice items. Sixteen mathematics experts and elementary school teachers constituted different panels and implemented two standard setting activities with different ways of feedback and item-grouping approaches. The important conclusions are summarized below: 1. Cut scores were gradually and logically increased in accordance with the achievement of basic, proficient, and advanced. However, item difficulty had a great impact on the judgments of panelists. Therefore, the necessity of multiple rounds, discussion and feedbacks in a multi-stage standard setting process was guaranteed. 2. With different types of feedback and item-grouping approaches, the impact of item difficulty on the interjudge or intrajudge consistency was decreased. Especially, the procedure with Reckase charts feedback had more significant effect on improving intrajudge and interjudge consistency. In addition, item-grouping in accordance with item difficulty could effectively control the inaccuracy of the standard. 3. Finally, the procedural, internal, and external evidences for the validity of performance standards were raised and proved. In fact, the resulting cut scores for 6th graders were more appropriate than those for 4th graders.