Deep indoor illumination estimation based on spherical gaussian representation with scene prior knowledge

Abstract

High dynamic range (HDR) illumination estimation from a single low dynamic range image is a critical task in the fields of computer vision, graphics and augmented reality. However, directly learning the full HDR environment map or parametric lighting information from a single image is extremely difficult and inaccurate. As a result, we propose a two-stage network approach for illumination estimation that integrates spherical gaussian (SG) representation with scene prior knowledge. In the first stage, a convolutional neural network is utilized to generate material and geometric information about the scene, which serves as prior knowledge for lighting prediction. In the second stage, we model indoor environment illumination using 128 SG functions with fixed center direction and bandwidth, allowing only the amplitude to vary. Subsequently, a Transformer-based lighting parameter regressor is employed to capture the complex relationship between the input images with scene prior information and its SG illumination. Additionally, we introduce a hybrid loss function, which combines a masked loss for high-frequency illumination with a rendering loss for improving the visual quality. By training and evaluating the lighting model on the created SG illumination dataset, the proposed method achieves competitive results in both quantitative metrics and visual quality, outperforming state-of-the-art methods.

Keywords

• Illumination estimation
• Spherical gaussian (SG)
• Prior knowledge
• Deep neural network
• Mixed/augmented reality