DRIFTNET-EnVACK: Adaptive Drift Detection in Cloud Data Streams With Ensemble Variational Auto-Encoder Featuring Contextual Network

Abstract

The adoption of cloud computing has been increasingly common across several industries in recent years and offers unparalleled flexibility and scalability in managing computational resources. However, the increasing reliance on cloud infrastructure also raises concerns regarding the monitoring and mitigation of drift, which pertains to unforeseen modifications inside the cloud environment. The aforementioned modifications possess the capacity to induce operational disruptions and can affect cloud provider reputation. This paper introduces a novel methodology for a proactive drift detection approach, referred to as the Ensemble of Variational Auto-encoder featuring Sequential Contextual Network Extended Kolmogorov-Smirnov Test (EnVACK) a Multi-Mode Variational Auto-encoder with Kolmogorov-Smirnov. It analyzes the data distribution using a non-parametric deep learning approach to create the concept drift map to address sudden and gradual drift challenges in the non-Gaussian cloud domain. This map gives a drift overview at each resource and combined level. This has shown a 96% f-score in the cloud domain an improvement of 4% when evaluated against benchmark studies executed in different domains. The present methodology has been specifically devised to warn and predict the occurrence of drift in synthetic and real-world non-Gaussian cloud usage traces.

Keywords

• CPU usage
• concept drift
• drift map
• drift detection
• KS
• memory usage