Micro-Cells Enabling Spatio-Angular Joint Attention for High-Resolution Light Field Editing

Abstract

Light Field (LF) processing is a critical step in advancing immersive experiences beyond two-dimensional (2D) images. While LF-based technology facilitates a smooth transition from 2D to three-dimensional (3D) media, processing LFs composed of numerous images remains challenging. Most existing research has provided only limited processing capabilities for low-resolution LFs. For instance, LF editing typically offers basic functions like color changes and object removal. This paper proposes micro-cell-based processing for spatially and angularly large-scale LFs. By partitioning and rearranging the LF into micro-cells, which are then used as input for a vision transformer (ViT), we enable spatio-angular joint attention that significantly enhances performance. The local attention applied to small-sized micro-cells allows for high-resolution LF processing without excessive computational costs. This paper utilizes this combination of micro-cells and the ViT for LF editing applications. Experimental results demonstrate that the proposed LF editing system delivers exceptional performance both quantitatively and qualitatively, enabling high-resolution LF processing with a variety of editing functions.

Keywords

• Light fields
• attention mechanisms
• image synthesis
• image reconstruction