Categorizing Data-Driven Methods for Test Scenario Generation to Assess Automated Driving Systems

Abstract

This survey aims to provide an overview of various methods for generating data-driven test scenarios for assessing automated driving systems (ADSs). The survey updates the overall process of scenario generation and categorizes the current methods using a systematic literature review of 64 studies identified between 2017 and 01/2023. Overall, we demonstrate that the data-driven scenario generation process should be updated by another process step, scenario fusion, leading to seven process steps: 1) scope definition, 2) primary data source selection, 3) primary data collection, 4) scenario identification, 5) scenario fusion, 6) scenario generation, and 7) scenario evaluation. “Scenario fusion” aims to fuse scenarios identified in different data sources for a better coverage of the ADSs’ operational design domains (ODDs) and a more comprehensive scenario description. Moreover, we introduce an improved definition for the representativity of test scenario catalogs, which helps improve the collection of traffic data using sampling plans. Also, we show that real driving and police accident data are the most commonly used data input sources. Besides, we illustrate that the ODD is often not defined. Finally, we discuss that the standardization of test scenario generation is difficult because most methods do not address specific ADSs and test environments, and do not provide standardized interfaces. Overall, we recommend comparing existing approaches using the same input data and researching the mutual supplementation of the existing methods. Finally, pre-defined case studies, further standardized terminology, and standards for test execution and evaluation can help speed up the standardization process.

Keywords

• Advanced driver assistance systems
• autonomous driving
• system validation
• vehicle safety