Survival Prediction for Non-Small Cell Lung Cancer Based on Multimodal Fusion and Deep Learning

Abstract

Non-small cell Lung Cancer (NSCLC) is one of the most common types of lung cancer, accounting for approximately 80% to 85% of lung cancer cases, and survival prediction in this cancer is an essential medical task. Traditional survival prediction methods, which only rely on demographic and clinical variables, fail to fully characterize patients’ pathological characteristics and clinical factors, thus limiting the prediction effect. With the development of medical imaging technology and genomic methods, survival prediction studies have ushered in new analytical perspectives in recent years. However, most of the existing advanced techniques only rely on one class and few classes of medical data, which do not comprehensively characterize patients’ pathological features and clinical conditions, and similarly limit the predictive effect. To this end, in this paper a new method is proposed to solve this problem, that is, using flexible interpretable graph structure to fuse and model the multimodal data (including clinical data and CT images, etc.) of patients, so as to solve the problem of fragmentation and one-sidedness among multi-class data of survival prediction. A new multi-modal fusion graph convolutional network (FGCN) is proposed according to the characteristics of multi-modal graph data. The main body of the network structure is composed of a SAGE graph convolution layer with the self-attention mechanism, which accelerates the convergence of the model compared to the traditional one and ensures that the risk predicted by the model is as close to the actual situation as possible. More importantly, this study is the first to introduce the TopKPooling strategy in this model to address the problems of feature redundancy and excessive noise features arising from multimodal data fusion and to reduce the model complexity. A large number of ablation and comparison experiments and analyses on public non-small cell lung cancer datasets show that the method proposed in this paper achieves better prediction results, with a C-index value of 0.76, which exceeds the currently known advanced techniques, and fully proves the effectiveness and superiority of the method in this paper.

Keywords

• Survival prediction
• multimodal fusion
• CT
• deep learning
• graph convolutional networks