Deep Reinforcement Learning Based Control Strategy for Voltage Regulation of DC-DC Buck Converter Feeding CPLs in DC Microgrid

Abstract

A DC microgrid’s tightly regulated DC/DC converter encounters significant challenges in voltage stability, primarily due to the negative incremental resistance of constant power loads (CPLs). Conventional controllers often struggle with load variations and changes in system parameters. Therefore, there has been growing interest in adaptive machine learning algorithms, such as Deep Reinforcement Learning (DRL), to improve voltage regulation. This paper presents an end-to-end DRL framework based on a modified Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The framework is designed to directly control power switches for regulating the voltage of a DC/DC buck converter that supplies power to CPLs. Real-time experiments were conducted using OPAL-RT to validate the approach under diverse load cycles and converter parameter changes.Comparative analysis against other DRL-based control strategies, including Deep Q-learning (DQN) and Deep Deterministic Policy Gradient (DDPG), demonstrated the superior static and dynamic voltage response of the proposed modified TD3 DRL controller, particularly in scenarios involving load and parametric variations.

Keywords

• Voltage stability
• micro grid
• constant power loads
• dc-dc buck converter
• deep Q-learning
• deep deterministic policy gradient