A Big Data-Driven Hybrid Model for Enhancing Streaming Service Customer Retention Through Churn Prediction Integrated With Explainable AI

Abstract

Customer churn prediction is a critical issue that streaming services face as retaining existing subscribers is vital to the success of the business. Creating reliable churn prediction models is important because the costs of acquiring new customers are usually higher than those involved in retaining existing ones. In this study, we propose a big data-driven hybrid model combining a deep neural network with a machine-learning model to efficiently forecast customer churn. Our proposed model uses Long Short-Term Memory (LSTM) with a Gated Recurrent Unit (GRU) to capture the trends in subscribers’ usage patterns over time. In addition, light gradient boosting (Light GBM) is used to leverage insights from sequential modeling along with original attributes to forecast churn. Moreover, feature selection techniques like Chi-squared testing and Sequential Feature Selection (SFS) are utilized to choose the optimum set of features for our proposed model. Furthermore, several individual models, including deep learning and traditional machine learning algorithms are also evaluated and compared with our proposed hybrid model. Additionally, the study illustrates model interpretations using Shapley Additive Explanations (SHAP) and Explainable Boosting Machine (EBM) which are used for identifying influential features in streaming services enhancing customer retention efforts. These techniques provide transparency into our proposed model’s forecasting, making them more actionable and understandable for decision-makers. Extensive experimental evaluation demonstrates the hybrid model achieves best-in-class performance with 95.60% AUC and 90.09% F1 score.

Keywords

• Artificial intelligence
• classification algorithms
• deep learning
• decision support systems
• explainable AI
• model interpretation