Coordinated Active Power-Frequency Control Based on Safe Deep Reinforcement Learning

Abstract

The continuous increase in renewables penetration poses a severe challenge to the frequency control of interconnected power grid. Since the conventional automatic generation control (AGC) strategy does not consider the power flow constraints of the network, the traditional approach is to make tentative generator power adjustments based on expert knowledge and experience, which is time consuming. The optimal power flow-based AGC optimization model has a long solution time and convergence issues due to its non-convexity and large size. Deep reinforcement learning has the advantage of “offline training and online end-to-end strategy formation”, which yet cannot ensure the security of artificial intelligence (AI) in power grid applications. A coordinated optimal control method is proposed for active power and frequency control based on safe deep reinforcement learning. First, the method models the frequency control problem as a constrained Markov decision process, and an agent is designed by considering various safety constraints. Then, the agent is trained using the example of East China Power Grid through continuous interactions with the grid. Finally, the effect of the agent and the conventional AGC strategy is compared. The results show that the proposed approach can quickly generate control strategies under various operating conditions, and can assist dispatchers to make decisions online.

Keywords

• coordinated power and frequency control
• artificial intelligence (ai)
• safe deep reinforcement learning
• constrained markov decision process
• agent