Accelerating Digital Twin Development With Generative AI: A Framework for 3D Modeling and Data Integration

Abstract

Digital twins (DTs) have been introduced as valuable tools for digitally representing physical objects or assets. However, developing comprehensive and accurate DTs remains challenging due to the complexity of adding diverse data sources, creating realistic models, and enabling real-time synchronization. In this paper, we propose a DT framework that uses Generative Artificial Intelligence (GenAI) techniques integrated into the DT development pipeline to address these challenges and accelerate the creation of these virtual representations. We demonstrate how 3D generative models utilizing pre-trained 2D diffusion models, and Large Language Models (LLMs) can automate and accelerate key stages of the DT development process, which include 3D modeling, data acquisition and integration, as well as simulation and monitoring. By providing a use-case scenario of a smart medical cooler box, we demonstrate the effectiveness of the proposed framework, highlighting the potential of GenAI to reduce manual effort and streamline the integration of DT components. In particular, we illustrate how it can accelerate the creation of 3D models for DTs from 2D images by using 2D-to-3D generative models. Additionally, we show the use of LLM-based agents in automating the integration of data sources with a DT and connecting physical devices with their virtual counterparts. Challenges related to computational scalability, data privacy, and model hallucinations are highlighted, which need to be addressed for the widespread adoption of GenAI in DT development.

Keywords

• Generative AI
• large language models
• 3D generative models
• diffusion models
• digital twin