A Multifunctional Tactile Sensory System for Robotic Intelligent Identification and Manipulation Perception

Abstract

Abstract Humans recognize and manipulate objects relying on the multidimensional force features captured by the tactile sense of skin during the manipulation. Since the current sensors integrated in robots cannot support the robots to sense the multiple interaction states between manipulator and objects, achieving human‐like perception and analytical capabilities remains a major challenge for service robots. Prompted by the tactile perception involved in robots performing complex tasks, a multimodal tactile sensory system is presented to provide in situ simultaneous sensing for robots when approaching, touching, and manipulating objects. The system comprises a capacitive sensor owning the high sensitivity of 1.11E‐2 pF mm−1, a triboelectricity nanogenerator with the fast response speed of 30 ms, and a pressure sensor array capable of 3D force detection. By Combining transfer learning models, which fuses multimodal tactile information to achieve high‐precision (up to 95%) recognition of the multi‐featured targets such as random hardness and texture information under random sampling conditions, including random grasp force and velocity. This sensory system is expected to enhance the intelligent recognition and behavior‐planning capabilities of autonomous robots when performing complex tasks in undefined surrounding environments.

Keywords

• interaction state perception
• lotus nanofiber
• multi‐feature recognition
• multimodal tactile
• MXene