Unsupervised Tuning for Drift Detectors Using Change Detector Segmentation

Abstract

Concept drifts can occur due to various factors such as changes in the environment or sensor degradation, posing significant challenges to machine learning systems by potentially skewing decision-making processes. Therefore, detecting drifts is essential to maintain the integrity and functionality of these systems. Automatic detectors based on statistical information are usually used to accomplish this task. Optimizing drift detectors through tuning is crucial for effective concept drift analysis. However, relying on user expertise or labeled data for supervised tuning can be challenging. In such cases, the use of unsupervised tuning methods becomes a suitable alternative to adapt detectors to evolving data distributions. To address this gap in the literature an unsupervised tuning method was proposed, leveraging a time series segmentation method. This innovative approach aims to alleviate the reliance on labeled data or user expertise traditionally associated with supervised methods, offering a more adaptive and automated means of tuning detectors. Our results demonstrate that our proposed approach outperforms the default configuration in most evaluated cases. Furthermore, we show that our approach can improve the hyperparameter tuning process when the type of drift is known including supervised methods such as Random Search tuning. By adopting our approach, we can achieve better performance in drift detection and improve the accuracy and reliability of systems that rely on this critical task.

Keywords

• Algorithm tuning
• concept drift
• signal drift
• drift detector