Analysis of Multiple Choice Tests Designed by Faculty Members of Kermanshah University of Medical Sciences

• Main Authors: Reza Pourmirza Kalhori, Farah Roushanpour, Mansour Rezaei, Katayoun Esmaeili
• Format: Article
• Language: English
• Published: Kermanshah University of Medical Sciences 2013-12-01
• Series: Educational Research in Medical Sciences
• Online Access: http://journals.kums.ac.ir/ojs/index.php/EduRMedS/article/view/1625/3001
• ISSN: 2252-0341; 2252-0465

Dear Editor Multiple choice tests are the most common objective tests in medical education which are used to assess the ind-ividual knowledge, recall, recognition and problem solving abilities. One of the testing components is the post-test analysis. This component includes; first, qualitative analysis of the taxonomy of questions based on the Bloom’s educational objectives and percentage of the questions with no structural problems; and second, the quantitative analysis of the reliability (KR-20) and indices of difficulty and differentiation (1). This descriptive-analytical study was aimed to qualitatively and quan-titatively investigate the multiple-choice tests of the faculty members at Kermanshah University of Medical Sciences in 2009-2010. The sample size comprised of 156 tests. Data were analyzed by SPSS-16 software using t-test, chi-squared test, ANOVA and Tukey multiple comparison tests. The mean of reliability (KR-20), difficulty index, and discrimination index were 0.68 (± 0.31), 0.56 (± 0.15) and 0.21 (± 0.15), respectively, which were acceptable. The analysis of the tests at Mashad University of Medical Sciences indicated that the mean for the reliability of the tests was 0.72, and 52.2% of the tests had inappropriate difficulty index and 49.2% of the tests did not have acceptable differentiation index (2). Comparison of the tests at Kermanshah University of Medical Sciences for the fields of anatomy, physiology, biochemistry, genetics, statistics and behavioral sciences courses at Malaysia Faculty of Medicine (3) and tests at Argentina Faculty of Medicine (4) showed that while difficulty index was acceptable in all three universities, but differentiation indices in Malaysia and Argentina Medical Faculties were higher than that in Kermanshah University of Medical Sciences. The mean for the questions with no structural flaws in all tests, taxonomy I, taxonomy II, and taxonomy III were 73.88% (± 14.88), 34.65% (± 15.78), 41.34% (± 13.14) and 23.99% (± 15.40), respectively. Comparison of the variances of qualitative and quantitative indices indicated significant differences between difficulty index, differ¬entiation index, reliability of the total test and percentage of taxonomy II among faculties (P_value<0.001), but this differences were not observed in taxonomies I and III. The results of Tukey multiple comparison test revealed a statistically significantly increase in the reliability of the medical faculty tests (P_value=0.001) and a statistically significantly decrease in the difficulty index of paramedi¬cal faculty tests compared to other faculties (P_value =0.041). Due to the lower differentiation index, the per¬centage of taxonomies II and III and the percentage of the questions with no structural problems compared to the standard criterion in some faculties, it is necessary to provide qualitative and quantitative feedback for the faculty members, as mentioned in previous studies (5) to promote their knowledge in designing the multiple-choice questions as an assessing tool of students.