Software Defects Identification: Results Using Machine Learning and Explainable Artificial Intelligence Techniques

Abstract

The rising deployment of software in automation and the cognitive skills of machines indicate a machine revolution in modern human civilization. Thus, diagnosing and predicting software faults is crucial to software reliability. In this paper, we first preprocessed four real datasets offered by National Aeronautics and Space Administration with twenty-one features using the Synthetic Minority Oversampling Technique and Label Encoding techniques. Subsequently, we experimented with thirteen software fault diagnosis Machine Learning (ML) models, i.e., Random Forest Regression, Linear Regression, Naïve Bayes, Decision Tree Classifier, Logistic Regression, KNeighbors Classifier, AdaBoost, Gradient Boosting Classifier, Gradient Boosting Regression, XGBR Regressor, XGBoost Classifier, Extra Trees Classifier and Support Vectors Machine after that, we compared each ML Model to select the best diagnostic model. Among them, XGBR outperformed, considering the accuracy, mean square error, and R2 score. We also used Explainable Artificial Intelligence (XAI), Local Interpretable Model (LIME), and SHapley Additive exPlanations (SHAP) to determine software fault features. We observed that Number of static invocations (nosi), Depth Inheritance Tree (dit), and Coupling Between Objects (cbo) features are the most affected software faults feature from datasets. For LIME, the average True positive of nosi is 40%, dit is 15%, and cbo is 20%; on the other hand, the SHAP average true positive value of nosi is 36%, cbo is 15%, and the norm true negative value of dit is 5%. Thus, LIME can afford the greatest impact on the model outcomes to identify features that are the most significant reasons for software defects.

Keywords

• Software defect prediction
• features selection
• software reliability
• software fault diagnosis
• explainable AI
• SHAP