From Data to Diagnosis: Enhancing Radiology Reporting With Clinical Features Encoding and Cross-Modal Coherence

Abstract

The integration of radiology reports for healthcare treatment using AI presents a transformative opportunity to enhance patient care and optimize healthcare delivery. Generating accurate radiology reports is crucial for guiding patient treatment in the clinical traditional applications or machines. However, the task of writing these reports can be a significant burden on medical practitioners. To address this challenge, we propose an adaptive, multi-modal technique for generating reports from chest X-rays. Our technique is based on the measurement that the content of clinical reports is highly accompanied by particular information present in the X-ray images. It comprises two key stages: Firstly, we introduce wisdom learning that leverages textual embedding to automatically extract and preserve medical wisdom from radiology reports. This wisdom base eliminates the need for manual labor in distilling valuable insights. Secondly, we employ a Cross-modal coherence technique to enhance the semantic alignment between reports, disease labels, and images. By utilizing clinical embedding, we channel the learning process of the visual feature attribute, ensuring meaningful associations between the different modalities. We conducted comprehensive evaluations of our model using both natural language generation metrics and clinical efficacy assessments on publicly available IU-Xray datasets. Through ablation analyses, we affirm the individual contributions of each module in improving the quality of generated reports. Moreover, when both modules are employed together, our approach surpasses existing state-of-the-art techniques across a wide range of evaluation metrics. This highlights the effectiveness of our technique in generating high-quality radiology reports and alleviating the burden on radiologists using intelligent medical applications or machines.

Keywords

• Clinical and visual features
• radiology reports
• encoder-decoder
• feature embedding
• wisdom learning