MGHE-Net: A Transformer-Based Multi-Grid Homography Estimation Network for Image Stitching

Abstract

Image stitching is one of the research hotspots in the fields of computer vision and image processing. Existing methods typically use traditional algorithms or deep learning-based algorithms to achieve this task. However, traditional image stitching algorithms perform poorly in images with weak textures, dark light and multiple noises. And the convolutional neural network (CNN) used by deep learning image stitching algorithms is difficult to capture the global contextual information of an image, resulting in limited accuracy. To address this issue, we designed a Multi-Grid Homography Estimation Network (MGHE-Net) based on Transformers. This network consists of cross-image integration feature extraction module, image matching module, and offset refinement module. The powerful global modeling capability of the Transformer is used to achieve multi-grid homography estimation from coarse to fine, improving the accuracy of image stitching. Experimental results demonstrate that our network not only achieves better stitching results in images with weak textures and dark light, but also reduces errors by 75.3% and 65.1%, respectively, compared to traditional algorithms and CNN-based algorithms on datasets with large parallax. Furthermore, our network improves the efficiency of image stitching.

Keywords

• Deep learning
• homography estimation
• image stitching
• MGHE-Net
• transformer