Instance-level medical image classification for text-based retrieval in a medical data integration center

Abstract

A medical data integration center integrates a large volume of medical images from clinical departments, including X-rays, CT scans, and MRI scans. Ideally, all images should be indexed appropriately with standard clinical terms. However, some images have incorrect or missing annotations, which creates challenges in searching and integrating data centrally. To address this issue, accurate and meaningful descriptors are needed for indexing fields, enabling users to efficiently search for desired images and integrate them with international standards.This paper aims to provide concise annotation for missing or incorrectly indexed fields, incorporating essential instance-level information such as radiology modalities (e.g., X-rays), anatomical regions (e.g., chest), and body orientations (e.g., lateral) using a Deep Learning classification model - ResNet50. To demonstrate the capabilities of our algorithm in generating annotations for indexing fields, we conducted three experiments using two open-source datasets, the ROCO dataset, and the IRMA dataset, along with a custom dataset featuring SNOMED CT labels. While the outcomes of these experiments are satisfactory (Precision of >75%) for less critical tasks and serve as a valuable testing ground for image retrieval, they also underscore the need for further exploration of potential challenges. This essay elaborates on the identified issues and presents well-founded recommendations for refining and advancing our proposed approach.

Keywords

• DICOM images
• Medical image captioning
• Medical image interchange
• SNOMED CT body structure