Applying Item Response Theory on the Nature Science Scholastic Achievement Test：Comparing Partial Credit Model and Graded Response Model

• Main Authors: Cheng-Chang Chen, 陳政漳
• Other Authors: Shu-Hui Cheng
• Format: Others
• Language: zh-TW
• Published: 2003
• Online Access: http://ndltd.ncl.edu.tw/handle/x83knr

碩士 === 中原大學 === 心理學研究所 === 91 === The purpose of the study was to compare Partial Credited Model and Graded Response Model of Item Response Theory to the Science Scholastic Achievement Test. The investigations focused on：（1）comparing dichotomous and polychotomous IRT models（2）assessing the model-data fitting among IRT models（3）comparing both item and ability indices between CTT and IRT。 One thousand examinees’ data were sampled from the testing data. There were 68 items in the science-SAT. They were consisted of three types of item type: (1)multiple-choice items (only one best answer), (2) multiple-choice items (more than one best answer) and (3) testlet. The major findings were： 1. The result of factor analysis show that the ratio of the first to the second eigenvalue was 3.956. The variation of first factor was 12.223. The data roughly met the assumption of unidimenionality. 2. All item parameters of dichotomous and polytomous models were estimated successfully excepted two items for 3PL model. The mean of item difficulty of 1PL was -0.10；the means of item difficulty and discrimination of 2PL were 0.05 and 0.85；the means of item difficulty, discrimination and guessing of 3PL were 0.48、1.47 and 0.2；the mean of difficulty of PCM when MMC（multiple multiple-choice）were scored as polytomous items was -0.03；the means of difficulty and discrimation of GRM when MMC were scored as polytomous items were 0.21 and 0.11；for MMC and testlet were scored in polytomous, the mean of difficulty of PCM was 0.00；the means of difficulty and discrimation of GRM were 0.13 and 0.85. 3. Three criteria were used to assess the goodness of fit in model selection：（1）chi-square and Q statistical indices：17 items were retained for the 1PL；47 items were retained for the 2PL；60 items were retained for the 3PL；4 and 3 items were retained for the PCM and GRM when MMC were scored as polytomous items；2 items were retained for the PCM and GRM separately when MMC and testlet were scored as polytomous items. The results showed 3PL can be applied to dichotomous data relatively well than other models.（2）item indices correlation between IRT models：correlation between difficulty of IRT models was high, range from 0.890∼0.999, the indices of discrimation were 0.133∼0.991.（3）regarding the test information , the 3PL provided more information than others. There was not much difference between 1PL and PCM. Same situation was occurred between 2PL and GRM. 4. Regarding the difficulty and discrimination of CTT, the mean of item difficulty and discrimination were 0.515 and 0.341. The correlation of difficulty between IRT models and CTT were -0.758∼-0.876, The correlation of discrimination were 0.166∼0.876. The result of indices between IRT and CTT revealed high correlations. The correlation between ability parameters of IRT models and total score of CTT ranged from 0.976 to 0.995. According to the analysis, 3PL is suitable for use in the science-SAT, the study hold conservative attitude to the feasibility of polytomous IRT models.