Design and tests of reinforcement-learning-based optimal power flow solution generator

Abstract

Optimal power flow (OPF) is a very traditional problem in the research field of power systems. In this paper, an OPF solution generator based on reinforcement learning (RL) is proposed. The solution process of OPF is modeled as a one-step Markov Decision Process (MDP) and is solved using the Twin Delayed Deep Deterministic policy gradient (TD3) algorithm. A warm-up training mechanism is adopted to realize better initialization of neural networks. Parallel computing is utilized to expand the searching range and improve training efficiency. Numerical tests are carried out in the IEEE-39 system. The results prove the correctness and efficiency of the proposed algorithm. The actor (policy) network of the well-trained agent can serve as a fast optimal power flow solution generator and can be applied to online scenarios.

Keywords

• Optimal power flow
• Reinforcement learning
• TD3
• Parallel computing