Real-Time Optimal Power Flow Using Twin Delayed Deep Deterministic Policy Gradient Algorithm

Abstract

The general concept of AC Optimal Power Flow (ACOPF) refers to the economic dispatch planning under electric network constraints. Moreover, each instance with the entire network must be solved in real-time (i.e., every five minutes) to ensure cost-effective power system operation while satisfying power balance equation. As the operation of power systems penetrated with intermittent renewable energy becomes more complicated, this article proposes Deep Neural Network (DNN) and Levenberg-Marquardt backpropagation-based Twin Delayed Deep Deterministic Policy Gradient (TD3) approach to improve computational performance of ACOPF. Specifically, because the ACOPF model shall consider prevailing constraints of the power system, including power balance equation, we set the appropriate reward vector in the training process to build our own policy. Furthermore, we add random Gaussian noise to individual net loads for representing uncertainty characteristics introduced by renewable energy sources. Finally, the proposed model is compared with the MAT-POWER solution on the IEEE 118-bus system to demonstrate its efficacy and robustness.

Keywords

• Deep deterministic policy gradient
• deep reinforcement learning
• Levenberg Marquardt
• optimal power flow
• twin delayed deep deterministic policy gradient