How I failed machine learning in medical imaging - shortcomings and recommendations

Medical imaging is an important research field with many opportunities for improving patients’ health. However, there are a number of challenges that are slowing down the progress of the field as a whole, such optimizing for publication. In this paper we reviewed several problems related to choosing datasets, methods, evaluation metrics, and publication strategies. With a review of literature and our own analysis, we show that at every step, potential biases can creep in. On a positive note, we also see that initiatives to counteract these problems are already being started. Finally we provide a broad range of recommendations on how to further these address problems in the future. For reproducibility, data and code for our analyses are available on https://github.com/GaelVaroquaux/ml_med_imaging_failures.

The great process in machine learning opens the door to many improvements in medical image processing (Litjens et al. , 2017; Cheplygina et al. , 2019; Zhou et al. , 2020). For example, to diagnose various conditions from medical images, ML algorithms have been shown to perform on par with medical experts (see Liu et al. , 2019, for a recent overview). Software applications are starting to be certified for clinical use (Topol, 2019; Sendak et al. , 2020).

Leave a Comment