Federated Learning With Dataset Splitting and Weighted Mean Using Particle Swarm Optimization

Abstract

Federated learning uses the concept of decentralized training of n number of local clients for a small number of epochs say 2-5, and then averaging the learned weights of all local clients, and evaluating on test dataset with the average weights loaded to a global model. The train dataset is split into n clusters and each cluster acts as a distributed data for each local model. Each round of weight averaging and then uploading the average weights on each local client for further training is called communication round and it was observed that similar accuracy can be obtained with a lesser amount of training time. In this paper, instead of averaging the weights, a weighted mean concept was developed where the PSO vector helps to find the weight values for the best accuracy of a global model. It was found that PSO can help in two ways by bettering the accuracy and also reducing the training time. The proposed approach can enhance the performance of pre-trained models like AlexNet, VGG16, InceptionV3, and ResNet50 on CIFAR-10 and CIFAR-100 datasets. The maximum increase was found with VGG16 of around 26.01% for CIFAR-10 and 26.84% for CIFAR-100. Similarly, on the Tomato dataset, AlexNet accuracy can be increased by 28.56%. Multi-modal model accuracy on the fake news dataset was also enhanced by 8.21%.

Keywords

• Federated learning
• particle swarm optimization
• optimization
• model performance
• multimodal