Actor-Critic reinforcement learning for optimal design of piping support constraint combinations

Abstract

Piping systems operating under various conditions may cause extremely unstable structural stress and deformation, leading to permanent deformation and damage to the piping structure, bringing about serious safety problems. So, piping stress analysis shall be performed along with proper design. Meanwhile, structural instability due to local stress concentration, which is one of the main interests of stress analysis, can be resolved through the appropriate selection of constraint conditions of piping supports at a relatively low cost. However, selecting an appropriate combination of constraints is time-consuming because analysis results are derived through many iterations. In addition, there is a big difference in the quality of the final design according to the experiences and capabilities of engineers. As a solution to this situation, this paper introduces a reinforcement learning strategy for deriving the optimal combination of constraints of piping supports. Primarily, we suggest applying the Actor-Critic(AC) learning algorithm for this problem, which is a type of reinforcement learning that has recently been in the spotlight for this problem because it can evaluate an action policy directly by the on-policy method inside the AC algorithm. During learning, it traces the correspondence between State and Action and suggests appropriate labeling criteria. We present the suitability and usefulness of the algorithm applied to the pipe stress analysis procedure through comparative verification between the results derived from the well-trained AC model and the results accomplished by engineers.

Keywords

• Actor-Critic (AC)
• Finite Element Method (FEM)
• Markov Decision Process (MDP)
• Markov Process (MP)
• Monte-Carlo (MC)
• Temporal-Difference (TD)