Medical inter-modality volume-to-volume translation

Abstract

Many clinical works require medical inter-modality imaging results since the supplementary imaging information from different modalities can be combined to provide better decision-making. Traditionally, this is done by scanning patients with different modalities, which is expensive, time-consuming, laborious, and may have health risks. Motivated by this problem, we propose a GAN-based method called W-VCT2VMRIGAN, which can automatically synthesize volumetric MRI from volumetric CT despite the presence of approximately %6 of imperfectly-paired slices, and thus can reduce cost, time, labor, and health risks caused by the traditional method. To show its effectiveness, we applied brain and pelvis datasets from clinical works to it. We also qualitatively and quantitatively compared it with the state-of-the-art techniques. The experimental result shows that in reference to the ground truth, our method outperforms the state-of-the-art Pix2Pix (12%, 15%, 260% better in average SSIM, average MS-SSIM3, MOS for brain; 12%, 9%, 230% better in average SSIM, average MS-SSIM3, MOS for pelvis), CycleGAN (30%, 24%, 520% better in average SSIM, average MS-SSIM3, MOS for brain; 42%, 56%, 680% better in average SSIM, average MS-SSIM3, MOS for pelvis), and MedSynthesisV1 (2%, 1%, 380% better in average SSIM, average MS-SSIM3, MOS for brain; 10%, 9%, 150% better in average SSIM, average MS-SSIM3, MOS for pelvis) techniques. Furthermore, we performed an ablation study for our method. The experimental result shows that in comparison to other variants, our method is optimal. Finally, we performed an experiment to choose the optimal hyperparameter regarding the number of epochs. The experimental result shows that the optimal number of epochs for brain and pelvis datasets are 900 and 400, respectively.

Keywords

• CT/MRI
• Generative adversarial networks
• Inter-modality
• Volume-to-volume