The Analysis of the Structural Problems of the Residency Promotion Test Items at Neurology Department, Kermanshah University of Medical Sciences in 2013

• Main Authors: Nazanin Razazian, Reza Pourmirza Kalhori
• Format: Article
• Language: English
• Published: Kermanshah University of Medical Sciences 2014-09-01
• Series: Educational Research in Medical Sciences
• Online Access: http://journals.kums.ac.ir/ojs/index.php/EduRMedS/article/view/1967/3692
• ISSN: 2252-0341; 2252-0465

Multiple-choice tests are the most common type of tests used to evaluate medical education. These tests have long been used in the written format for specialized boards and residency promotion tests in Iran and Kermanshah University of Medical Sciences (KUMS) (1). Multiple-choice tests are being analyzed using quanti-tative method (test level, difficulty level and discrim-ination level) and qualitative method (percentage of items free from structural problems and taxonomy of cognitive domain) (2). The most common method of analyzing structural problems of multiple-choice items is using Millman table which determines the construct validity of each item. Disregarding the testing principles by test makers can cause difficulty or ease of items, which directly affects the validity and difficulty and discrimination levels (3). This descriptive cross-sectional study was conducted in 2014 with an attempt to analyze the structural problems of 300 questions in two residency promotion tests (A & B test) at neurology department, KUMS. Data were obtained using the quantitative results of the test analysis software for special secretariat of promotion tests of ministry of health and medical education and report of the Medical Sciences Education Development Center at KUMS. The findings of the study indicated that 90% of the questions were free from structural problems in both tests. The validity of the whole test of neurology B, mean of difficulty index and mean of discrimination index were 0.92, 0.82 and 0.2, respectively. However in the case of neurology A test, these indices were 0.91, 0.74 and 0.22, respectively. The percentages of taxonomies I and II for neurology B and neurology A were 55.5% and 49.5%, respectively. Most structural problems of neurology B were incongruity of distracters (34%), editing errors (26.5%) and double negative (14%), respectively. In neurology A , however, most structural problems included non-determination of negative adverb in the stem (40%), incongruity of distracters (34%) and editing errors (14%). In comparison with the results of Shakournia et al. (2009), the structural problems in both studies were the same (3). The construction principles of multiple-choice questions are important to be seriously taken into account. Various studies have shown that the cognitive domain of the items is enhanced by improving the principles of test construction based on construct validity table (4). An important point found in the findings of this study was the difference of taxonomy level, difficulty level and structural errors in both tests, so that neurology B had been significantly constructed easier than neurology A; however, no difference was observed in quantitative indices. This confirms the necessity of continual qualitative assessment of residency promotion test items; especially construct validity based on structural construct validity table (5) in order to enhance their quality.