Assessing and comparing a DDPG model and GA optimization for a heat and power virtual power plant operating in a power purchase agreement scheme

Abstract

This paper proposes a deep deterministic policy gradient (DDPG) model for the operation management of a solar power-based virtual power plant (VPP) having a PPA with the grid and supplying power and thermal energy to consumers. The VPP serves to balance the solar power intermittency, cover the demand whenever solar power is absent, and ensure an efficient supply of energy. The literature in this field has introduced optimization algorithms to determine the power plant's output power or heat on a rolling-horizon basis. Using the function approximation category, which involves reinforcement learning with neural networks, to solve the simultaneous thermal and power operation management of VPPs is still not well developed. The challenges imposed in this model are sourced from the non-linearity of the CCHP, the power and thermal balance constraints, and the consideration of continuous variables rather than discrete ones. A case study is simulated in Egypt to assess and compare the models. Compared to the genetic algorithm optimization, the proposed DDPG model achieved 3% more profit, 12% higher carbon dioxide (CO2) emissions, and 9% lower natural gas consumption. The DDPG solution was 57% faster than the GA. The results of the DDPG model proved that machine learning methods could outperform optimization in terms of optimality achievement and speed of solution. The DDPG improved the operation of energy storage units and was able to recognize the supply-demand operational pattern, ensuring the scalability of the VPP to cope with different energy demand levels.

Keywords

• Machine learning
• Reinforcement learning
• Power purchase agreement
• Solar PV
• CCHP
• Virtual power plant