HVAC model-free optimal control method based on double-pools DQN

Abstract

In the field of HVAC (heating, ventilation and air conditioning) control, the model-based optimal control method has been extensively studied and verified by scholars, but this method highly depends on the accuracy of the model, the collection of a large amount of historical data, and the deployment of sensors.In response to the above problems,combined with EnergyPlus, actual system parameters and historical data, the HVAC optimized control model was constructed, and an improved double pools-based DQN (DPs-DQN) algorithm was proposed.Finally, it was applied to the load distribution of different types of chillers, the combined optimal control of cooling tower fan frequency and cooling water pump frequency in HVAC system.Based on the constructed problem model, aiming at the problem of sample imbalance in the decision-making optimization process, the algorithm established two independent experience pools on the basis of DQN to store load distribution and non load distribution samples respectively.During the training process, followed a certain ratio to sample from the experience pool to speed up the algorithm convergence.The proposed method was compared with the model-based control method and the baseline method.The experimental results show that compared with the baseline method, the model-based HVAC controller can save 11.5% (optimal energy-saving efficiency), while the DPs-DQN can save energy by 7.5% in the first year.At the same time, as the system runs, the controller can obtain results close to the optimal energy saving efficiency in the eighth year.In addition, compared with the model-based HVAC controller, the controller does not depend on the system model, and requires less prior knowledge and sensors in the online control process, which is more valuable in actual engineering applications.

Keywords

• deep reinforcement learning;model-free optimal control;HVAC system;building energy saving