Generation of a text description of weakly structured optical coherence tomography images

Abstract

Computer vision methods help to automate and improve processes in the field of medicine. In the field of ophthalmology, computer vision algorithms can be used to analyze images obtained using optical coherence tomography OCT, to identify pathologies and changes in the structure of the eye, however, due to the heterogeneity of patterns and configurations of tomographs, a comprehensive solution is needed. Within the framework of this work, an approach to the construction of a system for generating a text description of DICOM images using artificial intelligence methods is presented. To build a system of automatic description of anatomical properties and pathologies, a set of models for detection and classification was built on the OCT image. Data augmentation was performed for the task of recognizing areas with retinal slices in the OCT image. The computational experiment of constructing classification models showed recognition accuracy from 0.75 to 0.93 according to the balanced accuracy metric. Based on the developed models, a web service has been developed to demonstrate the functionality, which provides a report on finding 11 tags on an OCT scan.