Deep Reinforcement Learning Local Policy Transfer Method

Abstract

Purposes Reinforcement learning policy transfer is an effective way to reducing the consumption of deep reinforcement learning training. Local policy transfer is policy transfer at a fine-grained level, which is of great significance to the improvement of the global policy performance and the formation of a new global policy by the combination of local policies. Therefore, a deep reinforcement learning method for local policy transfer is proposed. Methods This method draws on the idea of “high cohesion, low coupling” in software engineering. By dividing the neural network, which is the carrier of policy, different sub-neural networks carry different local policies, and then realize the transfer of local policies through the transfer of sub-neural networks. This method supports flexible replacement and combination of local policies and forms a new global policy with better performance and adaption to new environment. In this paper, the classical deep reinforcement learning algorithm DQN is selected as the experimental algorithm and the transfer ability and performance of DQN algorithm before and after using the proposed method are compared. Findings The results show that the DQN algorithm realizes local policy transfer and improves its performance by about 27.5% after using the proposed method.

Keywords

• deep reinforcement learning
• local policy transfer
• dqn