Deep Reinforcement Learning Based Dynamic Proportional-Integral (PI) Gain Auto-Tuning Method for a Robot Driver System

Abstract

To meet the growing trend of stringent fuel economy regulations, automakers around the world are designing modules such as engines, motors, transmissions and batteries to be as efficient as possible. In order to verify the effect of these designs on the overall fuel efficiency of the vehicle, the vehicle equipped with each module is placed on the chassis dynamometer, driven to follow the target vehicle speed, and actual fuel efficiency is measured. These tests are traditionally performed by human operators, but are now being replaced by robots (physical or software) to ensure the accuracy and reliability of test results. Although the conventionally proposed proportional integral (PI)-based controller has a simple structure and is easy to implement, it requires the process of finding the optimal gain whenever the test conditions such as vehicle or drive cycle change, which is difficult and time consuming. In this study, we propose a proportional integral controller gain adjustment algorithm using deep reinforcement learning. The reinforcement learning agent learns to dynamically modify the PI gain value of the acceleration/deceleration pedal to better follow the target vehicle in a simulation environment. The perturbation is used in each training episode to reduce the difference between the simulation and real testing environment. Upon completion of the training process, the trained agent performs an adjustment process that generates a reference gain table. We then use this reference gain table to perform a real test. The performance of the proposed system was evaluated using Hyundai Tucson HEV (NX4) on an AVL chassis dynamometer. We also compared the performance of our proposed algorithm to traditional fuzzy logic-based PI controllers. The obtained experimental results show that the proposed control system achieved a performance improvement of aounrd 46.8% compared to the conventional PI control system in terms of root mean square error.

Keywords

• Automation
• deep Q-learning
• emission test
• machine learning
• PID control
• reinforcement learning