Unsupervised Real Time and Early Anomalies Detection Method for Sewer Networks Systems

Abstract

Sewer networks (SNs) are susceptible to various factors that can lead to failures, resulting in economic losses and environmental pollution. Data-driven approaches based on sewage flow monitoring enhance the awareness and maintenance capabilities of SNs. However, the current research lacks early warning systems for flow anomalies. This presents a challenge for the application of supervised methods, primarily due to the scarcity of anomalous flow datasets. Even with the availability of such datasets, the effectiveness of these methods may vary due to environmental differences, since SNs are situated in diverse environments. Therefore, effectively achieving early warnings for anomalies in unlabeled flow data is a challenge that must be addressed in the field of flow monitoring. To address this challenge, we propose a detection method for effectively warning of anomalies in flow data. Since anomalies typically result in significant deviations from normal data, early warnings can be achieved by comparing the differences between current and historical data. The key to this early warning lies in establishing an adaptive threshold for detecting abnormal data changes. Our detection method employs an unsupervised bagging-based multi-anomaly detection algorithm to detect such abnormal data changes. Experiments conducted on Erhai Lake SNs flow data demonstrate that our method can predict anomalies 5-15 minutes in advance with a precision of 80.00%, a recall of 66.67%, and an F1 score of 0.73. Our approach not only achieves cost-effective and timely anomalies detection but also overcomes the challenges associated with limited dataset availability, making it applicable to various other industries.

Keywords

• Unsupervised
• anomaly detection
• bagging
• sewer networks
• data-driven
• early warning