Intelligent End-Edge Computation Offloading Based on Lyapunov-Guided Deep Reinforcement Learning

Abstract

To address the end-edge computation offloading challenge in the multi-terminal and multi-server environment, this paper proposes an intelligent computation offloading algorithm based on Lyapunov optimization and deep reinforcement learning. We formulate a network computation rate maximization problem while balancing constraints including offloading time, CPU frequency, energy consumption, transmission power, and data queue stability. Due to the fact that the problem is mixed integer nonlinear programming, we transform it into a deterministic problem based on Lyapunov optimization theory, and then model it as a Markov decision process. Then, we employ deep reinforcement learning algorithm, i.e., asynchronous advantage actor-critic (A3C), and propose Lyapunov-guided A3C algorithm named LyA3C to approximate the optimal computation offloading policy. Experiments show that the LyA3C algorithm can converge stably and effectively improve the long-term network computation rate by 2.8% and 5.7% in comparison to the A2C-based and TD3-based algorithms.

Keywords

• edge computing
• computation offloading
• Lyapunov optimization
• deep reinforcement learning