Entropy to Mitigate Non-IID Data Problem on Federated Learning for the Edge Intelligence Environment

Abstract

Machine Learning (ML) algorithms process input data making it possible to recognize and extract patterns from a large data volume. Likewise, Internet of Things (IoT) devices provide knowledge in a Federated Learning (FL) environment, sharing parameters without compromising their raw data. However, FL suffers from non-independent and identically distributed (non-iid) data, which means it is heterogeneous data and has biased input data, such as in smartphone data sources. This bias causes low convergence for ML algorithms and high energy and bandwidth consumption. This work proposes a method that mitigates non-iid data through a FedAvg-BE algorithm that provides Federated Learning with the border entropy evaluation to select quality data from each device by cross-device in a non-iid data environment. Extensive experiments were performed using publicly available datasets to train deep neural networks. The experiment result evaluation demonstrates that execution time saves up to 22% for the MNIST dataset and 26% for the CIFAR-10 dataset, with the proposed model in Federated Learning settings. The results demonstrate the feasibility of the proposed model to mitigate non-iid data impact.

Keywords

• Big data
• collaborative learning
• federated learning
• machine learning
• non-IID