PCCN‐RE: Point cloud colourisation network based on relevance embedding

Abstract

Abstract The colour of the point cloud provides abundant information for perception tasks such as autonomous driving and virtual reality. However, few prior work studied the automatic colourisation of colourless point clouds. In this paper, the authors propose a novel method named as Point cloud Colorization Network based on Relevance Embedding (PCCN‐RE) relying on three structures: a relevance embedding structure that efficiently captures local information through the calculation of a covariance matrix within nearby points; a weighted pooling structure designed to facilitate the fusion of features; an enhanced spatial transform network structure that keeps the invariance of input point clouds. On the ShapeNetCore dataset, our PCCN‐RE generates more authentic colour than state‐of‐the‐art methods for colourless point clouds and achieves the highest results by obtaining a Peak Signal to Noise Ratio of 9.40 and a Structural Similarity Index of 0.62.