A real-time interactive restoration system for intraoral digital videos using segment anything model

Abstract

Objective Poor conditions in the intraoral environment often lead to low-quality photos and videos, hindering further clinical diagnosis. To restore these digital records, this study proposes a real-time interactive restoration system using segment anything model. Methods Intraoral digital videos, obtained from the vident-lab dataset through an intraoral camera, serve as the input for interactive restoration system. The initial phase employs an interactive segmentation module leveraging segment anything model. Subsequently, a real-time intraframe restoration module and a video enhancement module were designed. A series of ablation studies were systematically conducted to illustrate the superior design of interactive restoration system. Our quantitative evaluation criteria contain restoration quality, segmentation accuracy, and processing speed. Furthermore, the clinical applicability of the processed videos was evaluated by experts. Results Extensive experiments demonstrated its performance on segmentation with a mean intersection-over-union of 0.977. On video restoration, it leads to reliable performances with peak signal-to-noise ratio of 37.09 and structural similarity index measure of 0.961, respectively. More visualization results are shown on the https://yogurtsam.github.io/iveproject page . Conclusion Interactive restoration system demonstrates its potential to serve patients and dentists with reliable and controllable intraoral video restoration.