Energy Management Strategy of Integrated Electricity-Heat Energy System Based on Federated Reinforcement Learning

Abstract

The energy management of the electricity-heating integrated energy system (IES) is related to the economic benefits and multi-energy complementary capabilities of a park, but it faces the challenges of the randomness of renewable energy and the uncertainty of load. First, in this paper, a mathematical model of the energy management problem for the electricity-heating IES is conducted, and each energy supply subsystem is empowered as an agent. Based on the deep deterministic policy gradient (DDPG) algorithm, a system energy management model is established that comprehensively considers the real-time energy load of the subsystem, the time-of-use pricing, and the output of each equipment. Then, the federated learning technology is used to interact with the gradient parameters of the energy management model of the three subsystems during the training process to synergistically optimize the training effect of the model, which can protect the data privacy of each subsystem while breaking the data barriers. Finally, an example analysis verifies that the proposed federated-DDPG energy management model can effectively improve the economic benefits of the park-level IES.

Keywords

• integrated energy system (ies)
• federated learning
• energy management
• deep deterministic policy gradient (ddpg)