A Comparison of Multidimensional Item Selection Methods in Simple and Complex Test Designs

• Main Authors: Eren Halil ÖZBERK, Selahattin GELBAL
• Format: Article
• Language: English
• Published: EPODDER 2017-03-01
• Series: Journal of Measurement and Evaluation in Education and Psychology
• Subjects: Item selection method; multidimensional computer adaptive testing; multidimensional item response theory; composite score estimation
• Online Access: http://dergipark.gov.tr/epod/issue/28110/286956

In contrast with the previous studies, this study employed various test designs (simple and complex) which allow the evaluation of the overall ability score estimations across multiple real test conditions. In this study, four factors were manipulated, namely the test design, number of items per dimension, correlation between dimensions and item selection methods. Using the generated item and ability parameters, dichotomous item responses were generated in by using M3PL compensatory multidimensional IRT model with specified correlations. MCAT composite ability score accuracy was evaluated using absolute bias (ABSBIAS), correlation and the root mean square error (RMSE) between true and estimated ability scores. The results suggest that the multidimensional test structure, number of item per dimension and correlation between dimensions had significant effect on item selection methods for the overall score estimations. For simple structure test design it was found that V1 item selection has the lowest absolute bias estimations for both long and short tests while estimating overall scores. As the model gets complex KL item selection method performed better than other two item selection method.