A patch-based real-time six degrees of freedom object pose refinement method for robotic manipulation

Abstract

A fundamental vision technique for industrial robots involves the six degrees of freedom pose estimation of target objects from a single image. However, the direct estimation of the six degrees of freedom object pose solely from a single image is subject to limited accuracy. Various refinement approaches have been proposed to improve the accuracy by utilizing rendered images from a 3D model. Nevertheless, balancing speed and accuracy in an industrial setting remains a challenge for these methods. In this study, we propose a novel six degrees of freedom pose refinement approach centered around matching real image patches. Unlike previous approaches, our method does not rely on a 3D model, resulting in increased speed and accuracy. In the offline phase, we construct an offline database using image patches obtained from real images. During the inference phase, our method initially identifies the image patch within the offline database that is closest to the initial pose. Subsequently, we refine the six degrees of freedom pose by matching the corresponding image patches from the offline database. Experimental results indicate that our six degrees of freedom pose refinement method achieves real-time capability with a frame rate of 71 Frames Per Second (FPS), along with high precision. When the threshold is set to 0.5% of the object diameter, the average distance of dots score on the test data surpasses 70%. Moreover, experiments involving gripping and assembling tasks on an industrial robot demonstrate the ability of our method to autonomously select appropriate grasping angles and positions in real time. It further generates suitable motion paths, ultimately ensuring production efficiency.