Deep Learning-Based Engraving Segmentation of 3-D Inscriptions Extracted From the Rough Surface of Ancient Stelae

Abstract

Ancient stelae are considered important historical sources. However, it is a challenge to recognize the inscriptions carved on stelae that have rough surfaces due to prolonged weathering. In this paper, we propose a deep learning-based method to extract engraved regions from the 3D scanned mesh of a stela. First, the uneven distribution of vertices in the mesh is transformed using a mesh subdivision method such that the vertices in the mesh are uniformly distributed. Then, surface features (depth, concave features, and local surface features) are extracted from the subdivided mesh. The depth represents the basic shape of the mesh and is obtained from the aligned mesh. The concave features effectively represent concave regions by using a Frangi filter, and the local surface features have the spin image technique applied to describe the fine shapes of neighboring vertices relative to a vertex. The mesh and the surface features are rasterized into feature images, and engraved regions are segmented from the feature images by using a FC-DenseNet. Our experiments confirm that the proposed method effectively extracts engraved regions of the inscriptions from the rough surface of a stela and it shows robustness to noisy and extremely abraded characters. The proposed method outperformed the second-best method, obtaining an F1 score, IoU, and SIRI of approximately 2.95%, 3.65%, and 7.53%, respectively.

Keywords

• Cultural heritage
• relief extraction
• mesh processing
• engraving segmentation
• image segmentation