Combining 2D and 3D features to improve road detection based on stereo cameras

Abstract

Road detection is a fundamental component of autonomous driving systems since it provides validspace and candidate regions of objects for driving decision. The core of roaddetection methods is extracting effective and discriminative features. Sincetwo‐dimensional (2D) and 3D features are complementary, the authors propose arobust multi‐feature combination and optimisation framework for stereo imagepairs, called Feature++. First, several 2D and 3D features such as Gabor andplane are, respectively, extracted after the generation of 2D super‐pixel and a3D depth image from stereo matching. Second, the combined features are fed intoa three‐layer shallow neural network classifier to decide whether a super‐pixelis road region or not. Finally, the classified results are further refined usingfully connected conditional random field (CRF), taking the content informationinto consideration. We extensively evaluate the performance of four 2D features,four 3D features, and their combinations. Experiments conducted on the KITTIROAD benchmark show that (i) the combinations of 2D and 3D features greatlyimprove the road detection performance and (ii) using CRF as a refinement stepis necessary. Overall, their proposed ‘Feature + +’ method outperforms mostmanually designed features, and is comparable with state‐of‐the‐art methods thatare based on deep learning methods.

Keywords

• 3D features
• 2D features
• road detection
• stereo cameras
• autonomous driving systems
• driving decision