Digital Chemical Engineering (Dec 2022)

Application of a predictive Q-learning algorithm on the multiple-effect evaporator in a sugarcane ethanol biorefinery

  • Erick Y. Emori,
  • Mauro A.S.S. Ravagnani,
  • Caliane B.B. Costa

Journal volume & issue
Vol. 5
p. 100049

Abstract

Read online

With the recent development of machine learning, reinforcement learning is an interesting alternative to PID controllers. In this context, a discrete predictive Q-learning approach is applied in the control of a sugarcane biorefinery multiple-effect evaporation system. The algorithm is built using Scilab and learns to control the multiple-effect evaporator outlet concentration by manipulating its feed steam flow rate. Based on multiple episodes, the state-actions that consist of discrete changes in steam flow rate are chosen with a greedy algorithm. In order to increase the training efficiency and overcome the large dead time of the system, a neural network is applied to predict the outlet concentration of each control action after reaching the steady-state. The control policy was built and tested through simulations on a phenomenological model. The controller performance was evaluated in set-point tracking and disturbance rejection tests and compared with PID responses. The research showed that the Q-learning controller exhibited better performance than the PID controller.

Keywords