Discriminative analysis of schizophrenia patients using an integrated model combining 3D CNN with 2D CNN: A multimodal MR image and connectomics analysis

Abstract

Objective: Few studies have applied deep learning to the discriminative analysis of schizophrenia (SZ) patients using the fusional features of multimodal MRI data. Here, we proposed an integrated model combining a 3D convolutional neural network (CNN) with a 2D CNN to classify SZ patients. Method: Structural MRI (sMRI) and resting-state functional MRI (rs-fMRI) data were acquired for 140 SZ patients and 205 normal controls. We computed structural connectivity (SC) from the sMRI data as well as functional connectivity (FC), amplitude of low-frequency fluctuation (ALFF), and regional homogeneity (ReHo) from the rs-fMRI data. The 3D images of T1, ReHo, and ALFF were used as the inputs for the 3D CNN model, while the SC and FC matrices were used as the inputs for the 2D CNN model. Moreover, we added squeeze and excitation blocks (SE-blocks) to each layer of the integrated model and used a support vector machine (SVM) to replace the softmax classifier. Results: The integrated model proposed in this study, using the fusional features of the T1 images, and the matrices of FC, showed the best performance. The use of the SE-blocks and SVM classifiers significantly improved the performance of the integrated model, in which the accuracy, sensitivity, specificity, area under the curve, and F1-score were 89.86%, 86.21%, 92.50%, 89.35%, and 87.72%, respectively. Conclusions: Our findings indicated that an integrated model combining 3D CNN with 2D CNN is a promising method to improve the classification performance of SZ patients and has potential for the clinical diagnosis of psychiatric diseases.

Keywords

• Schizophrenia
• Brain connectivity
• Integrated model
• Convolutional neural network
• Classification