3D Deformable Object Matching Using Graph Neural Networks

Abstract

Considering the current advancements in computer vision it can be observed that most of it is focused on two-dimensional imagery. This includes problems such as classification, regression, and the lesser-known object matching problem. While object matching ca be viewed as a solved problem in a two-dimensional space, for a three-dimensional space there is a long way to go, especially for non-rigid objects. The problem is focused on matching a given object to a target object. We propose a solution based on Graph Neural Networks that tries to generalize over multiple objects at once, based on self-attention and cross-attention blocks for the network. To test our solution, we utilised five convolutional operators for the layers of the model. The convolutional operators we compared included GCNConv, ChebConv, SAGEConv, TAGConv, and FeaStConv. This paper aims to find the best operators for our architecture and the task. Our approach obtained favourable results for predicting the barycentric weights for the model, while struggling to predict the triangle indexes. The best results were obtained for the models using GCNConv, for the triangles index prediction and FeaStConv for the barycentric coordinates prediction. Received by the editors: 29 November 2023. 2010 Mathematics Subject Classification. 68T45, 68U05. 1998 CR Categories and Descriptors. I.2.10 Artificial Intelligence: Vision and Scene Understanding – 3D/stereo scene analysis; I.3.5 Computer Graphics: Computational Geometry and Object Modeling – Geometric algorithms, languages, and systems.

Keywords

• graph neural networks
• object matching
• 3D objects
• deformable objects