TFEdet: Efficient Multi-Frame 3D Object Detector via Proposal-Centric Temporal Feature Extraction

Abstract

This paper proposes the Temporal Feature Extraction Detector (TFEdet), a novel deep learning-based 3D multi-frame object detector efficiently utilizing temporal features from consecutive point clouds. To leverage previously processed frames, inter-frame bipartite matching is performed between current detections from a pre-trained single-frame detector and predicted prior detections, while considering the ego-motion. Subsequently, based on inter-frame association, two types of proposed temporal features are accumulated: temporal proposal features, which are aggregated single-frame features of proposals, and inter-frame proposal features, which containing explicit information between frames. These collected temporal features are then temporally encoded in a Gated Recurrent Unit (GRU)-based temporal feature extraction head and added as residuals to the current frame proposals, leading to the final detection. In performance evaluations on the nuScenes dataset, the proposed TFEdet, which processes a relatively smaller number of point clouds, handles more than twice the frames per second compared to previous multi-frame detectors and still demonstrates competitive detection performance through effective utilization of temporal proposal features.

Keywords

• 3D object detection
• multi-frame detection
• autonomous driving
• LiDAR point cloud
• gated recurrent unit