Real-time semantic segmentation for autonomous driving: A review of CNNs, Transformers, and Beyond

Abstract

Real-time semantic segmentation is a crucial component of autonomous driving systems, where accurate and efficient scene interpretation is essential to ensure both safety and operational reliability. This review provides an in-depth analysis of state-of-the-art approaches in real-time semantic segmentation, with a particular focus on Convolutional Neural Networks (CNNs), Transformers, and hybrid models. We systematically evaluate these methods and benchmark their performance in terms of frames per second (FPS), memory consumption, and CPU runtime. Our analysis encompasses a wide range of architectures, highlighting their novel features and the inherent trade-offs between accuracy and computational efficiency. Additionally, we identify emerging trends, and propose future directions to advance the field. This work aims to serve as a valuable resource for both researchers and practitioners in autonomous driving, providing a clear roadmap for future developments in real-time semantic segmentation. More resources and updates can be found at our GitHub repository: https://github.com/mohamedac29/Real-time-Semantic-Segmentation-Survey

Keywords

• Real-time semantic segmentation
• Transformer
• Deep learning
• Autonomous driving
• Neural network