IEEE Access (Jan 2024)
High Accurate Rephotographic Image Registration by Attention Masks: Enabling Intention-Driven Rephotographic Image Registration With Interactive Areas of Interest Masks
Abstract
Embarking on the journey of rephotography, capturing a contemporary image from the vantage point of a historical counterpart and registering them, is a formidable challenge. Traditional automated registration methods stumble in the face of this task, while manual methods, reliant upon painstakingly identified corresponding points, demand an investment of time, precision, and expertise. Often, only image fragments can be seamlessly registered due to changes in the scene, like new and removed buildings. Determining the areas of interest (AOI) for registration becomes a critical decision, placing users in the process’s role as curators. This work proposes a new method combining state-of-the-art automatic deep learning-based registration methods with user-provided masks. Users draw masks around the AOI they want to register and exclude non-indented AOI from registration. Using AOI masks reduces the required time, painstaking identification of corresponding points, and knowledge needed for manual registration while giving the user control over the registration process by providing an intuitive way to embed which AOI is vital to register. This interactive method achieves excellent registration quality and positive user feedback compared to regular automated image registration methods. It can not replace manual registration completely. However, for many rephotography tasks, it significantly reduces the required effort. The deep learning-based automatic method already achieves a high acceptance rate i.e., a score of at least 4 out of 5 of 55%, which is a considerable improvement to standard automatic registration method with an acceptance rate of 12%. With the interactive AOI masks method, which combines user-drawn masks with the automatic deep learning-based method, the acceptance rate increases to 60% and is almost as good as manual registration with a rate of 65%.
Keywords
