Optimal sequential decision making with probabilistic digital twins

Abstract

Abstract In this study, we present a formal definition of the probabilistic digital twin (PDT). Digital twins are emerging in many industries, typically consisting of simulation models and data associated with a specific physical system. In order to define probabilistic digital twins, we discuss how epistemic uncertainty can be treated using measure theory, by modelling epistemic information via $$\sigma$$ σ -algebras. A gentle introduction to the necessary mathematical theory is provided throughout the paper, together with a number of examples to illustrate the core concepts. We then introduce the problem of optimal sequential decision making. That is, when the outcome of each decision may inform the next. We discuss how this problem may be solved theoretically, and the current limitations that prohibit most practical applications. As a numerically tractable alternative, we propose a generic approximate solution using deep reinforcement learning together with neural networks defined on sets. We illustrate the method on a practical problem, considering optimal information gathering for the estimation of a failure probability.

Keywords

• Probabilistic digital twin
• Epistemic uncertainty
• Sequential decision making
• Partially observable Markov decision process
• Deep reinforcement learning