A Hierarchical Learning Approach to Autonomous Driving Using Rule Specifications

Abstract

Understanding the movement of surrounding objects and controlling robot platforms (such as autonomous vehicles and social robots) in a safe way are challenging problems. In the autonomous driving problem, autonomous vehicles must take into account the future behaviors of nearby vehicles and make appropriate controls accordingly. This is the biggest factor that makes autonomous driving problems difficult. In this work, this problem is tackled by combining benefits from both sequence prediction and deep reinforcement learning in a hierarchical manner. The driver’s behavior is classified according to the driving style defined by the rules selected in the autonomous driving situation. High-level behavior represents driving style, and the vehicle’s movement model is trained to condition itself to high-level behavior. Instead of directly finding low-level controls, we focus on finding high-level behaviors to increase efficiency. For example, if an autonomous vehicle needs to change lanes in certain situations, the high-level behavior “change lanes” is first selected and the corresponding vehicle movement model is used to find the appropriate low-level controls. Reinforcement learning is used to help select the best high-level behavior, and future behaviors of nearby vehicles are jointly reasoned to lead to better understanding of the current situation. The feasibility of the proposed approach is tested in publicly available datasets. The proposed method shows better efficiency and performance compared to existing learning-based control algorithms.

Keywords

• Autonomous driving
• reinforcement learning
• formal methods