Generalisability through local validation: overcoming barriers due to data disparity in healthcare

• Main Authors: Mitchell, William G. (Author), Dee, Edward C. (Author), Celi, Leo Anthony G. (Author)
• Other Authors: Massachusetts Institute of Technology. Institute for Medical Engineering & Science (Contributor)
• Format: Article
• Language: English
• Published: BioMed Central, 2021-12-02T15:04:32Z.
• Subjects: Article
• Online Access: Get fulltext

Abstract Cho et al. report deep learning model accuracy for tilted myopic disc detection in a South Korean population. Here we explore the importance of generalisability of machine learning (ML) in healthcare, and we emphasise that recurrent underrepresentation of data-poor regions may inadvertently perpetuate global health inequity. Creating meaningful ML systems is contingent on understanding how, when, and why different ML models work in different settings. While we echo the need for the diversification of ML datasets, such a worthy effort would take time and does not obviate uses of presently available datasets if conclusions are validated and re-calibrated for different groups prior to implementation. The importance of external ML model validation on diverse populations should be highlighted where possible - especially for models built with single-centre data.
National Institute of Health (Grant NIBIB R01 EB017205)