IEEE Transactions on Neural Systems and Rehabilitation Engineering (Jan 2023)

Alzheimer’s Disease Prediction via Brain Structural-Functional Deep Fusing Network

  • Qiankun Zuo,
  • Yanyan Shen,
  • Ning Zhong,
  • C. L. Philip Chen,
  • Baiying Lei,
  • Shuqiang Wang

DOI
https://doi.org/10.1109/TNSRE.2023.3333952
Journal volume & issue
Vol. 31
pp. 4601 – 4612

Abstract

Read online

Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer’s disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this work, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights into detecting AD-related abnormal neural circuits.

Keywords