DME: An Adaptive and Just-in-Time Weighted Ensemble Learning Method for Classifying Block-Based Concept Drift Steam

Abstract

This study proposes a novel incremental learning algorithm called distribution matching ensemble (DME) in context of adaptive weighted ensemble learning. In particular, DME estimates the distribution of each received data block by Gaussian mixture model (GMM) and reserves the corresponding distribution information, as well it maintains a group of classifiers in a buffer. When we receive a new data block which is required to be predicted, the similarity between its distribution and each reserved distribution will be calculated by Kullback-Leibler (KL) divergence, and then the similarities can be used to guide the weight assignment of each corresponding classifier to further make adaptive ensemble decision. DME gets rid of the underlying hypothesis that the most recent labeled data block always has the most similar distribution with the current unlabeled data block. In addition, to avoid infinite extension of ensemble buffer during incremental learning, we also develop two dynamic classifier update rules. Experiments results on some synthetic and real-world streaming datasets show that the proposed DME algorithm is able to track and adapt to various types of concept drift just in time. Especially, on data stream with frequent reoccurring drifts, the DME significantly outperforms to several state-of-the-art algorithms, indicating its superiority.

Keywords

• Data stream
• weighted ensemble learning
• concept drift
• Gaussian mixture model
• Kullback-Leibler divergence