FLAIR MRI sequence synthesis using squeeze attention generative model for reliable brain tumor segmentation

Abstract

Manual segmentation of brain tumors using structural magnetic resonance imaging (MRI) is an arduous and time-consuming task. Therefore, automatic and robust segmentation will considerably influence neuro-oncological clinical trials by reducing excessive manual annotation time. Herein, we propose a deep learning model that automatically segments brain tumors even in cases of missing MRI sequences, which are common in practical clinical settings. To address this issue, we enhance a generative adversarial network (GAN) by incorporating a squeeze-and-excitation (SE) attention module into its generator and a PatchGAN into its discriminator. The SE module recalibrates channel responses by explicitly modeling interdependencies, enabling the network to focus on critical regions such as tumor areas. Our proposed generative model is optimized using a combination of adversarial, structural similarity, and mean absolute error losses to synthesize missing MRI sequences more effectively. This enhancement allows our model to synthesize the missing MRI sequence (fluid attenuated inversion recovery [FLAIR]) by leveraging information from other available sequences (T1-weighted, T2-weighted, or contrast-enhanced T1-weighted [T1ce]). For the segmentation task, we employ an optimized nnU-Net model, which is trained using existing sequences and evaluated using both available and synthesized sequences (including missing ones), mimicking real-world scenarios where often only limited MRI sequences are available or usable. Our findings reveal a notable enhancement in brain tumor segmentation, as indicated by a significant increase in overall the Dice similarity coefficient (DSC) from 0.688% (when FLAIR is missing) to 0.873% (when using synthesized FLAIR derived from T2). This improvement brings the segmentation performance closer to what was achieved when real FLAIR was available, where the DSC reaches 0.901%. Moreover, our synthesizing model was also tested on two additional datasets: the BraTS 2020 validation set and BraTS Africa 2023 training set, which produces results comparable to those of BraTS 2021, thereby proving its robustness and generalizability. In addition, the resulting tumor segmentations are subsequently employed to assess the response to treatment in cases where all sequences were available and when synthesis was employed, according to response assessment in neuro-oncology criteria.

Keywords

• Brain tumor segmentation
• MR sequence synthesis
• NnU-net
• GANs
• Multi-contrast MR
• Deep learning