An Improved Q-Learning Algorithm and Its Application in Path Planning

Abstract

Traditional Q-Learning algorithm has the problems of too many random searches and slow convergence speed. Therefore, in this paper an improved ε-Q-Learning algorithm based on traditional Q-Learning algorithm was propased and applied to path planning. The key of this method is to introduce the dynamic search factor technology, which adjusts the greedy factor dynamically according to the feedback of the environment. If one exploration from the beginning to the end fails, the randomicity of the next exploration will be increased by increasing greedy factor, in order to avoid falling into the local optimization dilemma. Conversely, purpose will be increased by reducing greedy factor. The performance of the algorithm is evaluated by loss function, running efficiency, number of steps, and total return. Experiments show that compared with the existing Q-Learning algorithm, ε-Q-Learning can not only find a better optimal path, but also significantly reduce the cost of iterative searching.

Keywords

• path planning
• artificial intelligence
• reinforcement learning
• q-learning