Measuring change in knowledge acquisition of Rwandan residents: using the American Board of Pediatrics International In-Training Examination (I-ITE) as an independent tool to monitor individual and departmental improvements during the Human Resources for Health program: an observational study

• Main Authors: Natalie McCall, Christian Umuhoza, Cliff O’Callahan, Tanya Rogo, Diane Stafford, Aimable Kanyamuhunga, Peter T. Cartledge
• Format: Article
• Language: English
• Published: BMC 2019-06-01
• Series: BMC Medical Education
• Subjects: In-training examination; Formative feedback; Education, medical; Global Health; Rwanda; Internship and residency
• Online Access: http://link.springer.com/article/10.1186/s12909-019-1617-8

Abstract Background Rwanda is the only African country to use the pediatric International In-Training Examination (I-ITE). The objectives of this study were to use the scores from the I-ITE to outline the baseline level of knowledge of Rwandan residents entering the pediatric residency and the trends in knowledge acquisition from 2012 to 2018, during the Human Resources for Health (HRH) Program, an education partnership between the Rwanda Ministry of Health and a consortium of US universities. Methods A retrospective descriptive analysis of the I-ITE exam scores, taken by all Rwandan pediatric residents for five of the six academic years of the study period. Individual resident scores were weighted using the non-Rwandan I-ITE sites to minimise confounding from annual variations in exam difficulty. Statistical analysis included descriptives with ANOVA to compare variation in annual mean scores. Results Eighty-four residents took 213 I-ITE exam sittings over the five exam cycles. The mean weighted I-ITE score of all residents increased from 34% in 2013 to 49% (p < 0.001) in 2018. The 32-point gap between the mean US-ITE and Rwandan I-ITE score in 2012–2013 was reduced to a 16-point gap in 2017–2018. First year resident (PG1) scores, which likely reflect the knowledge level of undergraduate medical students entering the residency program, increased from 34.8 to 44.3% (p = 0.002) between 2013 and 2018. Conclusions The I-ITE is an independent, robust tool, measuring both learners and the institutional factors supporting residents. This is the first study to demonstrate that the I-ITE can be used to monitor resident knowledge acquisition in resource-limited settings, where assessment of resident knowledge can be a major challenge facing the academic medicine community. The significant increase in I-ITE scores between 2012 and 18 reflects the substantial curricular reorganisation accomplished through collaboration between Rwandan and US embedded faculty and supports the theory that programs such as HRH are highly effective at improving the quality of residency programs and undergraduate medical education.