Research on Improvement of the Click-Through Rate Prediction Model Based on Differential Privacy

Abstract

Click-through rate prediction is crucial in network applications such as recommendation systems and online networks. Designing feature extraction schemes to obtain features and modeling users’ click behavior are used to estimate the probability of users clicking on recommended items. The AutoInt model is a recent and effective research finding. It constructs combined features by referencing the multi-head attention mechanism but does not fully mine meaningful high-order cross-features and ignores user privacy protection. To address this problem, this study proposes the differential privacy bidirectional long short-term memory network (DP-Bi-LSTM-AutoInt) model, which is an improved AutoInt model. A bidirectional long short-term memory network is added after the embedding layer to deeply mine the nonlinear relationship between user click behaviors and construct high-order features. Further, differential privacy technology is adopted for user privacy protection, and the Gaussian mechanism is used to randomly perturb the gradient descent algorithm of the model. Using the Criteo dataset to conduct experiments, the experimental results show that the accuracy of the Bi-LSTM-AutoInt model proposed herein is improved by 0.65 % compared to the original AutoInt model. When the privacy budget is greater than 3.0, the accuracies of the DP-Bi-LSTM-AutoInt and Bi-LSTM-AutoInt models are nearly equivalent. However, the DP-Bi-LSTM-AutoInt model algorithm is more secure and reliable than the AutoInt model.

Keywords

• Click-through rate prediction
• differential privacy
• gradient descent
• long short-term memory