Using Markov Decision Process Model for Sustainability Assessment in Industry 4.0

Abstract

The manufacturing industry is facing increasing challenges to improve sustainability performance by using Industry 4.0 technologies like big data analytics, the internet of things, and digital twins, considering their potentials. However, effectively integrating sustainability goals and Industry 4.0 technologies within manufacturing systems can be challenging due to real-time attribute data of digital twins of machinery obtained by IoT tools and the necessity to integrate them into production planning levels. This challenge can be addressed by developing a framework for optimizing the flow of operations in a manufacturing system while incorporating sustainability assessment and Industry 4.0 technologies effectively. This paper investigates the integration of sustainability assessment considering Industry 4.0 technologies and the use of Markov Decision Process capabilities. The framework utilizes Markov Decision Process to model the decision-making process of the manufacturing system and its decision-makers. It includes sustainability goals as constraints or objectives in the Markov Decision Process model. The use of Industry 4.0 technologies is integrated into the framework to gather data and optimize the decision-making process based on that data. The capabilities of the proposed framework are demonstrated through case studies of a single agent on a shop floor. The findings from the case study indicate that the proposed framework can effectively support decision-making at the top-tier level of the enterprise by integrating sustainability assessment and the Industry 4.0 paradigm.

Keywords

• Industry 4.0
• Markov decision process
• sustainability assessment