Multimodal Neural Network for Recurrence Prediction of Papillary Thyroid Carcinoma

Abstract

Papillary thyroid carcinoma (PTC) is the most common endocrine carcinoma and has frequent recurrence instances. Although PTC recurrence has been predicted using predictors established using various features and techniques, its early detection is still challenging. To address this issue, it is aimed to develop a deep‐learning model that utilizes not only the initial medical records but also the thyroid function tests (TFTs) performed periodically post‐surgery. Herein, a novel multimodal prediction model, called the hybrid architecture for multimodal analysis (HAMA), that can analyze numeric and time‐series data simultaneously, is proposed. For quantitative evaluation, fourfold cross validation is conducted on data of 1613 PTC patients including 63 locoregional recurrence patients, and the HAMA is achieved the following performance: sensitivity (0.9688); specificity (0.9781); F1‐score (0.7943); and area under the receiver‐operating characteristic curve, AUROC (0.9863). Furthermore, a real‐time prediction simulation is conducted at 6‐month intervals by reconstructing the data of each patient into real‐time data. It is demonstrated in the real‐time simulation results that the HAMA predicts PTC recurrence at least 1.5 years in advance by recalculating the recurrence probability using the additional follow‐up data. To the best of the knowledge, the HAMA is the first deep‐learning model to reflect continuous change in the physical condition of a patient post‐surgery.

Keywords

• deep learning
• multimodal analysis
• papillary thyroid carcinoma
• recurrence prediction