Joint analysis of radiological reports and CT images for automatic validation of pathological brain conditions

Abstract

We consider a problem of validation of radiological medical reports and computed tomography images for an automated analysis of brain structures. Two methods for solving the problem are proposed: a method based on the ruCLIP multimodal model, and a method based on the joint use of two separate classifiers – for a text report and for a brain CT image. We discuss methods evaluation and the obtained results. The proposed approaches make it possible to correctly classify 99.6% of radiological reports from a test sampling into 15 possible diagnoses.

Keywords

• deep learning
• computed tomography
• computer-aided diagnosis
• pattern recognition
• natural language processing