Informatics in Medicine Unlocked (Jan 2024)
A recall-optimised machine learning framework for small data improves risk stratification for Hirschsprung's disease
Abstract
Objective: To improve the selection of patients with suspected Hirschsprung's disease for rectal biopsy with a recall-optimised machine learning framework that strongly penalises missed diagnoses, identify the most important clinical features, and investigate if this methodology is superior to the current practice of conventional logistic regression analysis. Study design: The study data comprised the medical records of 178 children older than one month who underwent a rectal biopsy for the evaluation of Hirschsprung's disease. Each medical record contained the clinical features recorded before the rectal biopsy and the biopsy result. However, only 20 of the 178 children were diagnosed with Hirschsprung's disease. Thus, as is frequently the case for rare diseases, the dataset was small and imbalanced. We present a machine learning framework that, based on these data, produces a champion machine learning model for predicting the presence of Hirschsprung's disease in patients. Notably, the machine learning framework is recall-optimised, i.e., it strongly penalises missed diagnoses. This is achieved using a combination of synthetic minority over-sampling techniques, Bayesian hyper-parameter optimisation, and bootstrapping-based prediction thresholding for a competing set of models comprised of a logistic regression model, a random forest classifier, and a gradient-boosted classifier. Finally, the machine learning framework evaluates the performance of these three competing models and conventional logistic regression with 5-fold stratified cross-validation using an 80:20 train-test split. Model performance is ranked by the average recall across the cross-validation folds. The model with the highest recall is selected as the champion. The model with the highest accuracy is chosen in case of a tie. Results: This study revealed that about 35% of our cohort's children without Hirschsprung's disease could have been spared a rectal biopsy while incurring no missed diagnoses using our machine learning framework. Given our aim of avoiding missed diagnosis, the champion model was vastly superior to conventional logistic regression, i.e., the status quo, which missed 40% of HD-positive cases. Moreover, the champion model pointed to a new hierarchy for the importance of clinical features associated with Hirschsprung's disease. Among all features, “gross abdominal distention” was the most important. Conclusion: There is considerable scope for a stricter selection when referring constipated children for a rectal biopsy to diagnose or exclude Hirschsprung's disease. This study demonstrates that a recall-optimised pipeline based on classical supervised machine learning is superior to the conventional statistical and heuristic approaches used today, also for small and imbalanced datasets. This finding opens a path to better care for patients with rare diseases while alleviating pressures on healthcare systems.
