Bayesian Deep Reinforcement Learning Algorithm for Solving Deep Exploration Problems

Abstract

In the field of reinforcement learning, how to balance the relationship between exploration and exploi-tation is a hard problem. The reinforcement learning method proposed in recent years mainly focuses on how to combine the deep learning technology to improve the generalization ability of the algorithm, but ignores the explo-ration-exploitation dilemma. The traditional reinforcement learning method can effectively solve the exploration problem, but there are certain restrictions: the state space of the Markov decision process must be discrete and limited. In this paper, the Bayesian method is proposed to improve the efficiency of deep reinforcement algorithm. And the main contribution is to extend the method of calculating the posterior distribution of parameters in Bayesian linear regression to nonlinear models such as artificial neural networks. By combining Bootstrapped DQN (deep Q-network) and the computational method proposed in this paper, Bayesian Bootstrapped DQN (BBDQN) is obtained. Finally, the results of the experiments in two environments show that BBDQN is more efficient than DQN and Bootstrapped DQN in the face of deep exploration.

Keywords

• deep reinforcement learning
• exploration and exploitation
• bayes theorem