BrainPixGAN: Generating intraoperative MRI images with mask-based generative networks

Abstract

In recent years, efforts to enhance precision in brain tumor surgeries have focused on using artificial intelligence (AI) with medical imaging. This involves integrating AI with medical imaging. This study aimed to generate a tumor-free MRI by using Generative Adversarial Networks (GANs) to establish a relationship between preoperative magnetic resonance imaging (MRI) and resection cavity segmentation masks obtained from intraoperative ultrasound (IOUS) data. For cavity segmentation, U-Net and U-Net with transfer learning were used, with the U-Net + EfficientNetB7 model achieving a high dice score of 97.82. The resection cavity mask was applied to preoperative MRI images using Pix2Pix, SPADE GAN, and BrainPixGAN. BrainPixGAN, incorporating transfer learning, outperformed the others, achieving SSIM 0.87, PSNR 35.89, and LPIPS 0.0037. This innovative approach represents a pioneering effort in generating GAN models for intraoperative MRI (iMRI) images using IOUS data, despite the challenges in setup and cost associated with iMRI imaging.

Keywords

• GANs
• Pix2Pix
• Spade
• Segmentation
• Brain tumor
• Brain neoplasm