Interactions Between Specific Human and Omnidirectional Mobile Robot Using Deep Learning Approach: SSD-FN-KCF

Abstract

To fulfill the tasks of human-robot interaction (HRI), how to detect the specific human (SH) becomes paramount. In this paper, the deep learning approach by the integration of Single-Shot Detection, FaceNet, and Kernelized Correlation Filter (SSD-FN-KCF) is developed. From the outset, the SSD is employed to detect the human up to 8m using the RGB-D camera with 320 × 240 resolution. Afterward the omnidirectional mobile robot (ODMR) is driven to the neighborhood of 2.5~3.0 m such that the depth image can accurately estimate the detected human's pose. Subsequently, the ODMR is commanded to the vicinity of 1.0m and the orientation inside -60~60° with respect to the optical axis to identify whether he/she is the SH by the FaceNet. To reduce the computation time of the FaceNet and extend the SH's tracking, the KCF is employed to achieve the task of HRI (e.g., human following). Based on the image processing result, the required pose for searching or tracking (specific) human is accomplished by the image-based adaptive finite-time hierarchical constraint control. Finally, the experiment with the SH, who is far from and on the backside of the ODMR, validates the effectiveness and robustness of the proposed approach.

Keywords

• Deep learning
• human detection
• face recognition
• visual tracking
• omnidirectional mobile robot
• adaptive finite-time hierarchical constraint control