CODE: A Moving-Window-Based Framework for Detecting Concept Drift in Software Defect Prediction

Abstract

Concept drift (CD) refers to data distributions that may vary after a minimum stable period. CD negatively influences models’ performance of software defect prediction (SDP) trained on past datasets when applied to the new datasets. Based on previous studies of SDP, it is confirmed that the accuracy of prediction models is negatively affected due to changes in data distributions. Moreover, cross-version (CV) defect data are naturally asymmetric due to the nature of their class imbalance. In this paper, a moving window-based concept-drift detection (CODE) framework is proposed to detect CD in chronologically asymmetric defective datasets and to investigate the feasibility of alleviating CD from the data. The proposed CODE framework consists of four steps, in which the first pre-processes the defect datasets and forms CV chronological data, the second constructs the CV defect models, the third calculates the test statistics, and the fourth provides a hypothesis-test-based CD detection method. In prior studies of SDP, it is observed that in an effort to make the data more symmetric, class-rebalancing techniques are utilized, and this improves the prediction performance of the models. The ability of the CODE framework is demonstrated by conducting experiments on 36 versions of 10 software projects. Some of the key findings are: (1) Up to 50% of the chronological-defect datasets are drift-prone while applying the most popular classifiers used from the SDP literature. (2) The class-rebalancing techniques had a positive impact on the prediction performance for CVDP by correctly classifying the CV defective modules and detected CD by up to 31% on the resampled datasets.

Keywords

• software defect prediction
• cross-version defect prediction
• chronological-defect data
• class rebalancing
• concept drift