Masked Structural Point Cloud Modeling to Learning 3D Representation

Abstract

Pre-training for 3D object recognition typically requires a large-scale 3D dataset to learn effective 3D geometric representations. However, constructing such datasets is costly due to the extensive 3D data collection and human annotation required. This paper explores a synthetic pre-training approach that learns 3D geometric representations by reconstructing structural point clouds without relying on real data or human annotation. We propose the Point Cloud Perlin Noise (PCPN) dataset, which is an automatically generated point cloud dataset that uses Perlin noise to simulate natural 3D structures found in the real world. The proposed method enables the rapid generation of diverse 3D geometric patterns using a simple Perlin noise-based formula, significantly reducing the human effort typically involved in creating conventional 3D datasets. We applied PointMAE to the PCPN dataset for pre-training, demonstrating improved performance in downstream tasks such as 3D shape classification and part segmentation. Our experiments showed that the proposed pre-trained model outperformed a model trained from scratch on ModelNet40 by 1.4%. In addition, our pre-training strategy proves effective for 3D object recognition without requiring real data or supervised labels. This study highlights that Perlin noise can capture 3D structural properties and that the diversity of geometric patterns is crucial for learning effective 3D geometric representations.

Keywords

• Deep learning
• computer vision
• 3D object recognition
• point cloud
• transfer learning
• self-supervised learning