Meta-Radiology (Jun 2025)

Large language model-based multi-source integration pipeline for automated diagnostic classification and zero-shot prognoses for brain tumor

  • Zhuoqi Ma,
  • Lulu Bi,
  • Paige Collins,
  • Owen Leary,
  • Maliha Imami,
  • Zhusi Zhong,
  • Shaolei Lu,
  • Grayson Baird,
  • Nikos Tapinos,
  • Ugur Cetintemel,
  • Harrison Bai,
  • Jerrold Boxerman,
  • Zhicheng Jiao

DOI
https://doi.org/10.1016/j.metrad.2025.100150
Journal volume & issue
Vol. 3, no. 2
p. 100150

Abstract

Read online

Purpose: In this study, we use large language models (LLMs) to integrate information from multi-source medical reports to enhance the accuracy of automated diagnostic classification and prognosis for brain tumors. Materials and Methods: Brain MRI reports from a cohort of 426 brain tumor patients were manually labeled for tumor presence and stability. Pathology reports from the same cohort were incorporated as an additional information source. A pre-trained LLM was used to extract features from the multi-source reports, and a Multi-layer perceptron (MLP) was trained for classification tasks. Model performance was evaluated on the test set using Micro F1 scores and AUROCs. The model’s zero-shot prognostic capability was validated on an independent cohort of 33 glioblastoma patients. Results: Micro F1-score 0.849 (95%CI: 0.814, 0.880) for tumor presence classification and 0.929 (95%CI: 0.904, 0.954) for tumor stability classification are reached. Compared to using solely radiology reports, the developed model showed improvements on Micro F1 of 10.4 ​% for tumor presence and 5.6 ​% for stability classification. Log-rank tests confirmed significant distinction between the high- and low-risk patient groups stratified by model-predicted “Tumor Stability” label (p-value ​= ​0.017), confirming the prognostic value of the model-generated labels. Conclusion: This study developed a multi-source integration model based on LLMs for automated diagnostic classification and zero-shot prognosis of brain tumors. The integration of multi-source reports improved classification accuracy compared to single-source reports. Predicted tumor stability labels demonstrated survival prognostic capabilities. These findings confirm the potential of LLMs in brain tumor research, supporting precision diagnostics and prognosis.

Keywords