Application of a predictive Q-learning algorithm on the multiple-effect evaporator in a sugarcane ethanol biorefinery

Abstract

With the recent development of machine learning, reinforcement learning is an interesting alternative to PID controllers. In this context, a discrete predictive Q-learning approach is applied in the control of a sugarcane biorefinery multiple-effect evaporation system. The algorithm is built using Scilab and learns to control the multiple-effect evaporator outlet concentration by manipulating its feed steam flow rate. Based on multiple episodes, the state-actions that consist of discrete changes in steam flow rate are chosen with a greedy algorithm. In order to increase the training efficiency and overcome the large dead time of the system, a neural network is applied to predict the outlet concentration of each control action after reaching the steady-state. The control policy was built and tested through simulations on a phenomenological model. The controller performance was evaluated in set-point tracking and disturbance rejection tests and compared with PID responses. The research showed that the Q-learning controller exhibited better performance than the PID controller.

Keywords

• Q-learning
• Second-generation ethanol
• Reinforcement learning
• Multiple-effect evaporation
• EMSO