Deep Q-Network for Optimal Decision for Top-Coal Caving

Abstract

In top-coal caving, the window control of hydraulic support is a key issue to the perfect economic benefit. The window is driven by the electro-hydraulic control system whose command is produced by the control model and the corresponding algorithm. However, the model of the window’s control is hard to establish, and the optimal policy of window action is impossible to calculate. This paper studies the issue theoretically and, based on the 3D simulation platform, proposes a deep reinforcement learning method to regulate the window action for top-coal caving. Then, the window control of top-coal caving is considered as the Markov decision process, for which the deep Q-network method of reinforcement learning is employed to regulate the window’s action effectively. In the deep Q-network, the reward of each step is set as the control criterion of the window action, and a four-layer fully connected neural network is used to approximate the optimal Q-value to get the optimal action of the window. The 3D simulation experiments validated the effectiveness of the proposed method that the reward of top-coal caving could increase to get a better economic benefit.

Keywords

• top-coal caving
• deep reinforcement learning
• deep Q-network
• discrete element method
• 3D-simulation