Advancing Mixed Reality Digital Twins Through 3D Reconstruction of Fresh Produce

Abstract

Access to safe and secure food is an essential human requirement for a sustainable world. Ensuring the safety of fresh food products necessitates accurate analytical techniques. This study presents an innovative food safety approach using rapid 3D reconstruction and digital twin analysis in mixed reality. While 3D models offer comprehensive insights into fresh food, conventional reconstruction methods are often complex due to multiple cameras or sensors. To address this, the research employs monocular depth estimation for detailed 3D reconstruction from a single camera view. This architecture uses web service protocols and the Global-Local Path Networks (GLPN) model for monocular depth perception, effectively translating 2D images into intricate 3D structures. These digital twins establish a strong foundation for enhanced analysis of food products, facilitating a better understanding of safety risks. The study also explores the applicability of transferring 3D models to Microsoft HoloLens 2, enabling immersive visualization and novel avenues for analysis. These techniques hold significance for the food industry, spanning scene comprehension, precision agriculture, robotics, augmented reality, and medical imaging. The methodology includes assessing surface degradation, with a focus on bruises on fresh produce. Employing a mesh overlay technique on the resulting 3D models distinctly showcases the impact of bruising. Sequential data recordings effectively track bruise expansion, shedding light on structural compromises. This technique dynamically portrays bruise progression, thus enriching comprehension of its implications on produce quality.

Keywords

• 3D reconstruction from 2D images
• monocular depth estimation
• global-local path networks (GLPN)
• point cloud filtering
• HoloLens integration
• digital twins