Flexible imputation toolkit for electronic health records

Abstract

Abstract Missing data in electronic health records (EHRs) poses a significant challenge for analysis. This study introduces Pympute, a comprehensive Python package designed for efficient and robust missing value imputation for EHRs. Pympute’s core algorithm, Flexible, intelligently selects the optimal imputation method for each variable based on its characteristics. Pympute offers a comprehensive suite of functionalities. It benchmarks the performance of ten existing machine learning imputation algorithms against Flexible on real-world EHR datasets containing laboratory measurements. Additionally, Pympute facilitates data simulation, generating realistic datasets mimicking real-world data distributions for controlled evaluation of imputation performance. Finally, Pympute investigates how missingness and skewness, influence the selection of optimal imputation algorithms within the Flexible framework. Our findings validate that Pympute’s Flexible method significantly improves imputation performance compared to the single model approach. Notably, simulating data solely based on covariance does not accurately reflect real-world selection behavior. Furthermore, skewness in the data distribution prompts Flexible to favor nonlinear imputation models. This study highlights the importance of considering data distribution patterns when selecting imputation algorithms. Pympute addresses this challenge by offering a versatile and user-friendly solution for diverse EHR data scenarios.

Keywords

• Electronic health records
• Missing data
• Data imputation
• Machine learning