I-NeRV: A Single-Network Implicit Neural Representation for Efficient Video Inpainting

Abstract

Deep learning methods based on implicit neural representations offer an efficient and automated solution for video inpainting by leveraging the inherent characteristics of video data. However, the limited size of the video embedding (e.g., 16×2×4) generated by the encoder restricts the available feature information for the decoder, which, in turn, constrains the model’s representational capacity and degrades inpainting performance. While implicit neural representations have shown promise for video inpainting, most of the existing research still revolves around image inpainting and does not fully account for the spatiotemporal continuity and relationships present in videos. This gap highlights the need for more advanced techniques capable of capturing and exploiting the spatiotemporal dynamics of video data to further improve inpainting results. To address this issue, we introduce I-NeRV, the first implicit neural-representation-based design specifically tailored for video inpainting. By embedding spatial features and modeling the spatiotemporal continuity between frames, I-NeRV significantly enhances inpainting performance, especially for videos with missing regions. To further boost the quality of inpainting, we propose an adaptive embedding size design and a weighted loss function. We also explore strategies for balancing model size and computational efficiency, such as fine-tuning the embedding size and customizing convolution kernels to accommodate various resource constraints. Extensive experiments on benchmark datasets demonstrate that our approach substantially outperforms state-of-the-art methods in video inpainting, achieving an average of 3.47 PSNR improvement in quality metrics.

Keywords

• video inpainting
• implicit neural representation
• random mask
• embedding