Image Inpainting of Thangka Murals Using Edge-Assisted Feature Fusion and Self Attention Based Local Refine Network

Abstract

Thangka murals hold significant cultural value in Tibet, but unfortunately, many of these precious murals have sustained damage over time. Therefore, the restoration of Thangka murals is crucial in preserving Tibet’s cultural heritage. The potential of Gated Convolution for Thangka mural restoration is significant due to its impressive performance. However, existing Gated Convolution-based approaches for Thangka restoration face two challenges: 1) restoration results using Gated Convolution still show blurring at the pixel level, and 2) Gated Convolution remains dependent on the perceptual field of the convolution during image inpainting. To address two challenges, we propose a novel Thangka mural inpainting method that incorporates two key components: 1) an Edge-assisted Feature Fusion (EFF) method that leverages edge information to assist in inpainting the texture details of the image and enhance the consistency of textures and edges; and 2) a Self Attention based Local Refine (SALR) network that effectively enhances the perceptual field of the convolution and captures long-distance dependencies. The experimental results demonstrate that our proposed method is effective in repairing broken areas of Thangka murals, even on a small dataset (N = 5605). To validate the effectiveness of our proposed architecture and its components, we conduct a series of ablation experiments. The results demonstrate the necessity of our approach and highlight the contributions of each component. Moreover, the proposed end-to-end approach exhibits broad applicability and can be extended to other profiling tasks based on small datasets.

Keywords

• Image inpainting
• Thangka restoration
• feature fusion
• self attention
• gated convolution