NeRFlex: Flexible Neural Radiance Fields With Diffeomorphic Deformation

Abstract

Due to the vast array of NeRF-based techniques, the representation power of Neural Radiance Fields (NeRF) has been quickly rising in recent years. However, it is still difficult to offer fresh perspectives for user-controlled geometry alterations with current techniques; instead, they restrict edits to those that were observed during training or to particular regions chosen by the provider. In this paper, we present NeRFlex, Flexible Neural Radiance Fields with Diffeomorphic Deformation, which lets users process geometry completely unrestricted by using radiance field deformation. Given a condition with a specific viewpoint, a conditional score function is estimated using diffusion time steps. The deformation between the radiance fields before and after an edit is then generated by the score function and the initial radiance field. To guarantee topology preservation, invertibility, and smooth transformation, diffeomorphic constraints are provided to the deformation field. Experiments on various objects demonstrate NeRFlex’s ability to generate flexible deformations and high-quality novel views after geometry edits, which were never observed in the training data. Continuous deformation along the pathway leading to the deformed object is also obtained by diffeomorphism decomposition.

Keywords

• NeRF
• neural radiance fields
• implicit neural fields
• diffeomorphic deformation