GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents

Abstract

To solve the problem that intelligent devices equipped with deep reinforcement learning agents lack effective security data sharing mechanisms in the intelligent Internet of things, a general federated reinforcement learning (GenFedRL) framework was proposed for deep reinforcement learning agents.The joint training through model-sharing technology was realized by GenFedRL without the need to share the local private data of deep reinforcement learning agents.Each agent device’s data and computing resources could be effectively used without disclosing the privacy of its private data.To cope with the complexity of the real communication environment and meet the need to accelerate the training speed, a model-sharing mechanism based on synchronization and parallel was designed for GenFedRL.Combined with the model structure characteristics of common deep reinforcement learning algorithms, general federated reinforcement learning algorithm suitable for single network structure and multi-network structure was designed based on the FedAvg algorithm, respectively.Then, the model sharing mechanism among agents with the same network structure was implemented to protect the private data of various agents better.Simulation experiments show that common deep reinforcement learning algorithms still perform well in GenFedRL even in the harsh communication environment where most data nodes cannot participate in training.

Keywords

• intelligent Internet of things;federal learning;federal reinforcement learning;deep reinforcement learning