Visual Route Recognition in Urban Spaces: A Scalable Approach Using Open Street View Data

Abstract

This article presents a novel pipeline for visual route recognition (VRR) in large-scale urban environments, leveraging open street view data. The proposed approach aims to identify the path of a video recorder by analyzing visual cues from continuous video frames and street landmarks, evaluated through datasets from New York and Taipei City. The pipeline begins with semantic visual geo-localization (SemVG), a semantic fused feature extraction network that filters out nonlandmark noise, generating robust visual representations. We construct a feature database from multiperspective street view images to enable efficient feature retrieval for query video frames. In addition, we introduce a spatio-temporal trajectory reconstruction method that corrects mismatches in the camera's motion path, ensuring consistency. Our contributions include the development of SemVG, a method for maintaining spatio-temporal consistency in trajectory reconstruction, and a large-scale Taipei dataset designed for VRR. This work has implications for urban surveillance, law enforcement, and smart city applications, supporting urban planning, resource management, search and rescue, and augmented reality navigation by improving localization without specialized hardware.

Keywords

• Metric learning
• street view imagery
• trajectory reconstruction
• visual geo-localization (VG)
• visual route recognition (visual route recognition)