Research on coal mine XR intelligent operation and maintenance system for complex collaborative tasks involving multiple humans and multiple robots

Abstract

With the development of coal mine intelligence and the application of coal mine robots, an efficient collaboration between coal mine operators and coal mine robots plays a crucial role in the execution of complex underground tasks. To optimize the complex operational relationship of multiple coal mine operators and multiple robots, based on the concept of digital twin and extensive experience in the XR field, the research is conducted on the design and key technologies of XR intelligent operation and maintenance system for complex collaborative tasks involving multiple humans and multiple robots in coal mines. Firstly, for a typical scenario of collaboration between two types of coal mine operators (i.e central control operators and field control operators) and two types of coal mine robots (i.e. detection robots and operating robots) in complex tasks, the overall system architecture is designed. The system is divided into three parts: the physical subsystem, VR operation and maintenance subsystem, and AR operation and maintenance subsystem. The content, functions, and collaborative operation relationships among these three parts are introduced. Then, an analysis of key technologies related to the VR operation and maintenance subsystem, AR operation and maintenance subsystem, and communication networking is carried out. The solutions corresponding to each key technology are discussed, and the integration and operation of the two types of coal mine operators, two types of coal mine robots, and VR/AR operation and maintenance subsystem are implemented. Finally, a laboratory environment simulating complex underground conditions is set up to create a test site, where the task points and specific tasks are defined. The feasibility and effectiveness of the system are tested and verified. The experimental results show that the coal mine XR intelligent operation and maintenance system is able to function in collaborative situations between multiple humans and multiple robots corresponding to different complex tasks. Through the collaborative operation of the VR operation and maintenance subsystem and the AR operation and maintenance subsystem, the collaborative perception, decision-making, and control between virtual space and physical space can be achieved. This allows for the iterative optimization and verification of complex tasks in a physical space from a virtual space, forming an intelligent operational mode of human-human, human-robot, and robot-robot interactive collaboration.

Keywords

• coal mine intelligence
• coal mine robots
• coal mine operators
• human-robot collaboration
• xr intelligent operation and maintenance
• digital twin