Machine learning with multimodal data for COVID-19
Weijie Chen,
Rui C. Sá,
Yuntong Bai,
Sandy Napel,
Olivier Gevaert,
Diane S. Lauderdale,
Maryellen L. Giger
Affiliations
Weijie Chen
Medical Imaging and Data Resource Center (MIDRC), USA; Division of Imaging, Diagnostics, and Software Reliability, Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, USA; Corresponding author. Medical Imaging and Data Resource Center (MIDRC), USA.
Rui C. Sá
Medical Imaging and Data Resource Center (MIDRC), USA; Department of Medicine, University of California, San Diego, USA
Yuntong Bai
Medical Imaging and Data Resource Center (MIDRC), USA; Division of Imaging, Diagnostics, and Software Reliability, Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, USA
Sandy Napel
Medical Imaging and Data Resource Center (MIDRC), USA; Department of Radiology, Stanford University, USA
Olivier Gevaert
Medical Imaging and Data Resource Center (MIDRC), USA; Department of Medicine and Department of Biomedical Data Science, Stanford University, USA
Diane S. Lauderdale
Medical Imaging and Data Resource Center (MIDRC), USA; Department of Public Health Sciences, University of Chicago, USA
Maryellen L. Giger
Medical Imaging and Data Resource Center (MIDRC), USA; Department of Radiology, University of Chicago, USA
In response to the unprecedented global healthcare crisis of the COVID-19 pandemic, the scientific community has joined forces to tackle the challenges and prepare for future pandemics. Multiple modalities of data have been investigated to understand the nature of COVID-19. In this paper, MIDRC investigators present an overview of the state-of-the-art development of multimodal machine learning for COVID-19 and model assessment considerations for future studies. We begin with a discussion of the lessons learned from radiogenomic studies for cancer diagnosis. We then summarize the multi-modality COVID-19 data investigated in the literature including symptoms and other clinical data, laboratory tests, imaging, pathology, physiology, and other omics data. Publicly available multimodal COVID-19 data provided by MIDRC and other sources are summarized. After an overview of machine learning developments using multimodal data for COVID-19, we present our perspectives on the future development of multimodal machine learning models for COVID-19.
