Federated Learning Based on Kernel Local Differential Privacy and Low Gradient Sampling

Abstract

Federated learning that is an approach to addressing the “data silo” problem in a collaborative fashion may face the risk of data leakage in real-world contexts. To solve this problem, we introduce the random Fourier feature mapping (RFFM) together with kernel local differential privacy (KLDP) and develop a new privacy protection mechanism, called the RFFM-KLDP mechanism, for high-dimensional context data. Theoretical properties show that the proposed privacy-preserving mechanism has the properties of $\epsilon $ -LDP and $\epsilon $ -distance-LDP in the federated learning framework. To guarantee the effectiveness of federated learning in the presence of contaminated data, we develop a modified low-gradient sampling technique to sample representative subset of uncontaminated data by incorporating large gradients and unbalanced information. By combining RFFM-KLDP and modified low-gradient sampling technique, we develop a novel and robust federated learning method for classification in the presence of the noisy text data, which can preserve data privacy and largely improve the accuracy of classification algorithm compared to the existing classifiers in terms of the area under curve and classification accuracy. Simulation studies and a context example are used to illustrate the proposed methodologies.

Keywords

• Differential privacy
• federated learning
• low gradient sampling
• random Fourier feature mapping