Unsupervised novelty detection for time series using a deep learning approach

Abstract

In the Smart Homes and IoT devices era, abundant available data offers immense potential for enhancing system intelligence. However, the need for effective anomaly detection models to identify and rectify unusual data and behaviors within Smart Home Systems (SHS) remains a critical challenge. This research delves into the relatively unexplored domain of novelty anomaly detection, particularly in the context of unlabeled datasets. Introducing the novel DeepMaly method, this approach provides a practical tool for SHS developers. Functioning seamlessly in an unsupervised manner, DeepMaly distinguishes between seasonal and actual anomalies through a unique process of training on unlabeled pristine features extracted from time series data. Leveraging a combination of Long Short-Term Memory (LSTM) and Deep Convolutional Neural Network (DCNN), the model is primed to detect anomalies in real-time. The research culminates in a comprehensive data prediction and classification process into normal and abnormal data based on specified anomaly thresholds and fraction percentages. Notably, this function operates seamlessly unsupervised, eliminating the need for labeled datasets. The study concludes with a complete data forecasting and sorting method that divides data into normal and abnormal categories based on defined anomaly thresholds and fraction percentages. Working in an unsupervised mode reduces the requirement for labeled datasets. The results highlight the model's prowess in new detection, which has been successfully applied to benchmark datasets. However, there is a restriction since deep learning algorithms can recognize noise as abnormalities. Finally, the investigation enhances SHS anomaly detection, providing a crucial tool for real-time anomaly identification in the ever-changing IoT and Smart Homes scene.

Keywords

• Anomaly detection
• SHS
• LSTM
• DCNN