Application of Deep Reinforcement Learning for Proportional–Integral–Derivative Controller Tuning on Air Handling Unit System in Existing Commercial Building

Abstract

An effective control of air handling unit (AHU) systems is crucial not only for managing the energy consumption of buildings but ensuring indoor thermal comfort for occupants. Although the initial control schema of AHU is appropriate at installation and testing, it is frequently necessary to adjust the control variables due to the changing thermal response of the building envelope and space usage. This paper presents a novel optimization process for the control parameters of old AHU systems in existing commercial buildings without system downtime and massive operational data. First, calibrating the building and system simulator with limited system operation data and unknown building parameters can provide identical responses to the system operation with the Hooke–Jeeves algorithm during the cooling season. The deep deterministic policy gradient algorithm is employed to determine the optimal control parameters for the valve opening position of the cooling coil within less than three hours of training based on the calibrated simulator. By using actual implementations with the developed optimal control variables for an old AHU in a real building, the proposed auto-tuned PID control in the simulator and with machine learning improves thermal environments with a steady room temperature (23.5 ± 0.5 °C) by 97% in occupied periods. It is also proved that this can reduce cooling energy consumption by up to 13.71% on a daily average. The successful AHU controller can improve not only the stability of AHU systems but the efficiency of a building’s energy use and indoor thermal comfort.

Keywords

• auto-tuned PID control
• air handling unit
• deep deterministic policy gradient (DDPG) algorithm
• virtual simulator
• Hooke–Jeeves algorithm
• existing commercial building