Multimodal Video Analysis for Crowd Anomaly Detection Using Open Access Tourism Cameras

Abstract

In this article, we propose the detection of crowd anomalies through the extraction of information in the form of time series in video format using a multimodal approach. Through pattern recognition algorithms and segmentation, informative measures of the number of people and image occupancy are extracted at regular intervals, which are then analyzed to obtain trends and anomalous behaviors. Specifically, through temporal decomposition and residual analysis, intervals or specific situations of unusual behaviors are identified, which can be used in decision-making and the improvement of actions in sectors related to human movement such as tourism or security. This methodology introduces a novel, privacy-focused approach by analyzing anonymized metrics rather than tracking or recognizing individuals, setting a new standard for ethical crowd monitoring. Applied to the webcam of Turisme Comunitat Valenciana in the town of Morella (Comunitat Valenciana, Spain), this approach has shown excellent results, correctly detecting specific anomalous situations and unusual overall increases during the previous weekend and during the October 2023 festivities. These results have been obtained while preserving the confidentiality of individuals at all times by using measures that maximize anonymity, without trajectory recording or person recognition.

Keywords

• anomaly detection
• multimodal analysis
• time series
• open data