Vertical handover policy for cyber-physical systems aided by SAGIN based on deep reinforcement learning

Abstract

The vertical handover policy of space-air-ground integrated cyber-physical systems based on deep reinforcement learning was studied, in which the challenges of complicated network model and difficulties in acquiring prior knowledge for network topology and model were addressed. By jointly taking the system stability, handover cost and network-using cost into account, the vertical handover policy problem was modeled as a constraint Markov decision process (CMDP), and a sufficient condition to ensure the existence of a feasible solution was derived.Furthermore, a constraint-proximal policy optimization (CPPO) algorithm was proposed to solve the CMDP, and also the distributed learning scheme at base station sides was introduced to accelerate the speed of converging. Simulation results verify the validation and superiority of the proposed vertical handover policy as compared with the baselines.

Keywords

• space-air-ground integrated network;cyber-physical system;deep reinforcement learning;vertical handover