UeDiff-GAN-based target detection and twin synchronization mapping for fully mechanized mining faces

Abstract

The construction of a digital twin model for fully mechanized mining faces requires manually creating a digital twin 3D model of a physical entity, performing target detection on the entity, and adjusting the 3D model based on real-time detection results to ensure synchronization mapping between the twin and the physical entity. Therefore, real-time and accurate detection of underground targets is crucial for achieving virtual-physical synchronization mapping control. Current mainstream target detection methods require incorporating or modifying modules within traditional models, resulting in complex network structures and prolonged training cycles, which reduce the real-time performance of target detection. Moreover, these methods struggle with precisely detecting targets in images with high-intensity noise. To address these issues, this study proposed a UeDiff-GAN-based target detection and twin synchronization mapping method for fully mechanized mining faces. The diffusion model was used to add noise into high-quality samples to generate samples of varying levels, which were then used to train a generative adversarial network (GAN) model. A smooth diffusion algorithm was designed to regulate the diffusion step sizes, while an imbalanced diffusion module was incorporated to obtain a detection algorithm model that matches pre-identified samples. A 3D model of the fully mechanized mining face was constructed and rendered using Unity3D, achieving a digital twin of underground physical entities. Based on this model, a mapping relationship between the physical entity and its twin model was established. The corresponding machine's motion state and posture are controlled according to the detection results at different underground locations. This approach enabled twin model coordinated control, thereby achieving process-level twinning. Experimental results on a self-developed dataset demonstrated that the UeDiff-GAN model improved the average detection accuracy of underground moving targets by 19.4%, 14.3%, 9.1%, and 24.3% compared to SSD, R-CNN, YOLOv7, and Diff-GAN models, respectively. The detection speed improved by 13.86, 42.73 frames per second (fps) compared to SSD and R-CNN models, respectively. The real-time delay between the twin model and the physical entity was at a maximum of 0.873 seconds.

Keywords

• fully mechanized mining face
• digital twin
• target detection
• virtual-physical synchronization mapping
• diffusion model
• gan