Advanced Zero-Shot Learning (AZSL) Framework for Secure Model Generalization in Federated Learning

Abstract

Federated learning (FL) introduces new perspectives in machine learning (ML) by enabling model training across decentralized devices. The research on data security and privacy in federated learning (FL) gaining popularity nowadays. However, FL has several challenges for instance, dealing with unseen classes, learning heterogeneous, non-independent, and identically distributed (non-IID) data distributions across the distributed devices, and most important model generalization without compromising privacy. Zero-Shot Learning (ZSL) and synthetic data are used traditionally to address these challenges. However, model generalization, privacy preservation with diverse and non-IID data distribution, and handling unknown classes remain unsolved. Thus, to solve these challenges, herein we propose an advanced framework named Advanced Zero-Shot Learning (AZSL). The proposed framework is based on the ZSL concept and creates synthetic data sources efficiently. The proposed AZSL utilizes the Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) for generating synthetic data and an EfficientNet-B7 as the global model. The proposed framework contributes to privacy preservation since data is stored locally on the devices while only sharing the model’s parameters. The objective of the proposed model is to increase model generalizability while achieving better performance on various and heterogeneous datasets while preserving data privacy. We have used datasets named MNIST, CIFAR-10, and FEMNIST to measure the model’s accuracy, precision, and generalization capability. The experimental results show that the proposed AZSL improves the results in terms of performance and accuracy. The deployment of WGAN-GP for synthetic data generation and EfficientNet-B7 for model training led to 4.8% to 7.4%. The proposed framework improved the ability to generalize to unseen classes, with a generalization gap reduction of up to 13.3% on CIFAR-10, and provided promising performance in handling non-IID data distribution. The proposed AZSL successfully resolved some central issues of FL, such as data privacy, generalization capability, and non-IID data. Incorporating AZSL and synthetic data generation also improved the model’s flexibility and effectiveness across the different datasets. The results show that the proposed framework can be helpful in practice across various domains that demand high data privacy.

Keywords

• Federated learning
• zero-shot learning
• synthetic data
• data privacy
• model generalization