Multimodal predictive classification of Alzheimer's disease based on attention‐combined fusion network: Integrated neuroimaging modalities and medical examination data

Abstract

Abstract Early diagnosis of Alzheimer's disease (AD) plays a key role in preventing and responding to this neurodegenerative disease. It has shown that, compared with a single imaging modality‐based classification of AD, synergy exploration among multimodal neuroimages is beneficial for the pathological identification. However, effectively exploiting multimodal information is still a big challenge due to the lack of efficient fusion methods. Herein, a multimodal fusion network based on attention mechanism is proposed, in which magnetic resonance imaging (MRI) and positron emission computed tomography (PET) images are converted into feature vectors with the same dimension, while the demographic information and clinical data are preprocessed and converted into feature vectors through embedding. This attention model can focus on important feature points, fuse the multimodal information more effectively, and thus provide accurate diagnosis and prediction for different pathological stages. The results show that the model achieves an accuracy of 84.1% for triple classification tasks in normal cognition (NC) versus mild cognitive impairment (MCI) versus AD and 93.9% prediction accuracy in stable MCI (sMCI) versus progressive MCI (pMCI). In contrast to the existing multimodal diagnosis methods, our model yields a state‐of‐the‐art accuracy of AD diagnosis, which is powerful and promising in clinical practice.

Keywords

• Alzheimer's disease
• attention mechanism
• diagnosis
• multi‐modal
• prediction