A Novel Reinforcement Learning-Based Particle Swarm Optimization Algorithm for Better Symmetry between Convergence Speed and Diversity

Abstract

This paper introduces a novel Particle Swarm Optimization (RLPSO) algorithm based on reinforcement learning, embodying a fundamental symmetry between global and local search processes. This symmetry aims at addressing the trade-off issue between convergence speed and diversity in traditional algorithms. Traditional Particle Swarm Optimization (PSO) algorithms often struggle to maintain good convergence speed and particle diversity when solving multi-modal function problems. To tackle this challenge, we propose a new algorithm that incorporates the principles of reinforcement learning, enabling particles to intelligently learn and adjust their behavior for better convergence speed and richer exploration of the search space. This algorithm guides particle learning behavior through online updating of a Q-table, allowing particles to selectively learn effective information from other particles and dynamically adjust their strategies during the learning process, thus finding a better balance between convergence speed and diversity. The results demonstrate the superior performance of this algorithm on 16 benchmark functions of the CEC2005 test suite compared to three other algorithms. The RLPSO algorithm can find all global optimum solutions within a certain error range on all 16 benchmark functions, exhibiting outstanding performance and better robustness. Additionally, the algorithm’s performance was tested on 13 benchmark functions from CEC2017, where it outperformed six other algorithms by achieving the minimum value on 11 benchmark functions. Overall, the RLPSO algorithm shows significant improvements and advantages over traditional PSO algorithms in aspects such as local search strategy, parameter adaptive adjustment, convergence speed, and multi-modal problem handling, resulting in better performance and robustness in solving optimization problems. This study provides new insights and methods for the further development of Particle Swarm Optimization algorithms.

Keywords

• PSO algorithm
• convergence
• diversity
• reinforcement learning
• Q-learning
• Q-table