Federated Double Deep Q-Learning-Based Computation Offloading in Mobility-Aware Vehicle Clusters

Abstract

On the edge side of internet of vehicles (IoV), mobile edge computing (MEC) servers with certain computational resources are deployed to provide computational service for vehicles. However, with the generation of a series of vehicle computing intensive and delay sensitive applications such as virtual reality (VR) navigation and vehicle-mounted games, the quality of service (QoS) for vehicle applications cannot be guaranteed with the limited computational resources in MECs. In this article, adjacent vehicles in IoV are converged into clusters based on topological stability. The idle computational resources of vehicles within the cluster are integrated and scheduled uniformly to serve vehicles within the cluster with computation task requirements. This reduces the dynamic changes in idle computational resources caused by rapid vehicle movement, and effectively ensured reliable feedback of computing results after offloading computing tasks. In order to reduce the task completion time, the task is decomposed into multiple subtasks with time dependencies, which can be processed in parallel. A directed acyclic graph (DAG) is used to characterize the task model. To further minimize latency in completing computing tasks, a federated double deep Q network-based computation task offloading (FDTO) strategy is proposed for vehicle clusters. The optimal computation task offloading decision is obtained based on the collaborative training and updating of double deep Q network (DDQN). Unlike traditional machine learning algorithms that require centralized training, federated learning (FL) allows only the training model parameters to be passed between vehicles, which not only protects data privacy but also reduces communication overhead. Simulation results show that the proposed strategy can effectively reduce task completion latency compared to the benchmark strategies.

Keywords

• Internet of Vehicles
• directed acyclic graph
• DDQN
• federated learning