Improving Kidney Tumor Classification With Multi-Modal Medical Images Recovered Partially by Conditional CycleGAN

Abstract

The accurate classification of kidney tumors necessitates the utilization of various diagnostic techniques. Within the domain of medical imaging tests, the integration of multi-modal medical imaging represents an innovative approach to enhance diagnostic precision. However, the integration of multi-modal medical imaging faces a critical challenge: the insufficiency of correspondingly paired data across modalities, resulting in a paucity of training samples for neural networks. To mitigate this limitation, generative artificial intelligence, specifically generative models capable of generating additional data, thereby addressing the gap in the multi-modal medical imaging record. In our work, our primary objective is to improve the classification results that outperform an existing method of using single-modal medical images. To achieve this, we harness the wealth of information of multi-modal data derived from Contrast-Enhanced Computed Tomography (CECT) and Magnetic Resonance Imaging (MRI), along with their respective subtypes, to determine which specific modality or pair contributes the most effectively to classification. Our work introduces the comprehensive comparison between the different multi-modal fusion techniques, in which the Area Under the Curve (AUC) serves as the benchmark for performance evaluation. Moreover, this work tackles the problem of the unavailability of kidney tumor data by partially recovering from the available data using Conditional CycleGAN, which is part of the image-to-image translation that maps between two different image domains. Through the employment of multi-modal fusion techniques and our proposed recovery of missing data, our research has yielded superior classification results than single-modal classification approaches.

Keywords

• Kidney tumor
• deep learning
• multi-modal
• image-to-image translation
• GANs