Multimodal Registration of Fundus Images With Fluorescein Angiography for Fine-Scale Vessel Segmentation

Abstract

Background and objective: The analysis of retinal vessels in fundus images is vital in the diagnosis of retinal diseases and early diagnosis of chronic vascular diseases and diabetes. Automatic vessel segmentation relies on costly expert annotations that may have limitations in the level of detail. We develop an automatic method to generate highly-accurate vessel segmentation including fine-scale vessels. Methods: We present a new framework for fine-scale vessel segmentation from fundus images through registration and segmentation of corresponding fluorescein angiography (FA) images. We first register and aggregate the extracted vessels highlighted from the fluorescent dye in the FA frames. This FA vessel mask is then registered to the fundus image based on an initial fundus vessel mask. Post-processing is performed to refine the final vessel mask. Registration of the FA frames, and registration of FA vessel mask to the fundus image, are performed by similar coarse-to-fine hierarchical frameworks comprising both projective and deformable registration. Two convolutional neural networks with identical network structures, both trained on public datasets but with different configurations, are used for vessel segmentation of both the FA frames and the fundus images. Results: Qualitative examples support the robustness and accuracy of the proposed method. Quantitative evaluations, including the area-under-curve (AUC) of the receiver operating characteristic (ROC) curve are presented. Although fair comparisons cannot be made due to a lack of similar methods and adequate public datasets, we demonstrate that the proposed method with an AUC ROC of 0.979, outperforms a state-of-the-art automatic vessel segmentation method trained on publicly available datasets at 0.956. Conclusions: The proposed method generates accurate vessel segmentation results containing filamentary vessels that are virtually indiscernible to the naked eye in color retinal fundus images.

Keywords

• Fundus images
• fluorescein angiography
• multi-modal registration
• vessel segmentation