Deep Reinforcement Learning-Based Tie-Line Power Adjustment Method for Power System Operation State Calculation

Abstract

Operation state calculation (OSC) provides safe operating boundaries for power systems. The operators rely on the software-aid OSC results to dispatch the generators for grid control. Currently, the OSC workload has increased dramatically, as the power grid structure expands rapidly to mitigate renewable source integration. However, the OSC is processed with a lot of manual interventions in most dispatching centers, which makes the OSC error-prone and personnel-experience oriented. Therefore, it is crucial to upgrade the current OSC in an automatic mode for efficiency and quality improvements. An essential process in the OSC is the tie-line power (TP) adjustment. In this paper, a new TP adjustment method is proposed using an adaptive mapping strategy and a Markov Decision Process (MDP) formulation. Then, a model-free deep reinforcement learning (DRL) algorithm is proposed to solve the formulated MDP and learn an optimal adjustment strategy. The improvement techniques of “stepwise training” and “prioritized target replay” are included to decompose the large-scale complex problems and improve the training efficiency. Finally, five experiments are conducted on the IEEE 39-bus system and an actual 2725-bus power grid of China for the effectiveness demonstration.

Keywords

• Operation state calculation
• tie-line power adjustment
• deep reinforcement learning
• stepwise training
• prioritized target replay