Deep imitation reinforcement learning for self‐driving by vision

Abstract

Abstract Deep reinforcement learning has achieved some remarkable results in self‐driving. There is quite a lot of work to do in the area of autonomous driving with high real‐time requirements because of the inefficiency of reinforcement learning in exploring large continuous motion spaces. A deep imitation reinforcement learning (DIRL) framework is presented to learn control policies of self‐driving vehicles, which is based on a deep deterministic policy gradient algorithm (DDPG) by vision. The DIRL framework comprises two components, the perception module and the control module, using imitation learning (IL) and DDPG, respectively. The perception module employs the IL network as an encoder which processes an image into a low‐dimensional feature vector. This vector is then delivered to the control module which outputs control commands. Meanwhile, the actor network of the DDPG is initialized with the trained IL network to improve exploration efficiency. In addition, a reward function for reinforcement learning is defined to improve the stability of self‐driving vehicles, especially on curves. DIRL is verified by the open racing car simulator (TORCS), and the results show that the correct control strategy is learned successfully and has less training time.

Keywords

• gradient methods
• reinforcement learning
• automobiles
• traffic engineering computing