Gradient descent Sarsa(?)algorithm based on the adaptive potential function shaping reward mechanism

Abstract

In the reinforcement leaning tasks with continuous state spaces,the algorithms are usually facing the problems of ill initial performance and low convergence speed.In order to solve these problems,the potential function shaping reward mechanism was proposed to improve the reinforcement learning algorithms.This mechanism propagates model knowledge to the learner adaptively in the form of the additional reward signal,so that the initial performance and convergence speed could be improved effectively.In view of the good performance and existing problems of the radial basis function (RBF) network,the adaptive normalized RBF (ANRBF) network was put forward to use as a potential function to generate the shaping rewards.A gradient descent (GD)algorithm named ANRBF-GD-Sarsa(?) was proposed based on the ANRBF network.The convergence of ANRBF-GD-Sarsa(?) algorithm was analyzed theoretically.Extensive experiments are conducted to show the good initial performance and high convergence speed of the proposed algorithm.

Keywords

• reinforcement learning;Sarsa(?);gradient descent;potential function;shaping reward