Q-Learning-Based Task Offloading and Resources Optimization for a Collaborative Computing System

Abstract

Mobile edge computing (MEC) can effectively overcome the shortcomings of high-latency in mobile cloud computing (MCC) by deploying the cloud resources, e.g., storage and computational capability, to the edge. However, the limited computation capability of the MEC restricts the scalability of offloading. Therefore, the basic requirements of the MEC system are to explore effective offloading decisions and resource allocation methods. To address it, we develop a collaborative computing system composed of local computing (mobile device), MEC (edge cloud) and MCC (central cloud). Based on the proposed collaborative computing system, we design a novel Q-learning based computation offloading (QLCOF) policy to achieve the optimal resource allocation and offloading scheme by prescheduling the computation side for each task from a global perspective. Specifically, we first model the offloading decision process as a Markov decision process (MDP) and design a state loss function (STLF) to measure the quality of experience (QoE). After that, we define the cumulation of STLFs as the system loss function (SYLF) and formulate an SYLF minimization problem. Due to the difficulty to directly solve the formulated problem, we decompose it into multiple subproblems and preferentially optimize the transmission power and computation frequency of the edge cloud by the quasi-convex bisection and polynomial analysis method, respectively. Based on the precalculated offline transmission power and edge cloud computation frequency, we develop a Q-learning based offloading (QLOF) scheme to minimize the SYLF by optimizing offloading decisions. Finally, the numeral results show that the proposed QLOF scheme effectively reduces the SYLF under different parameters.

Keywords

• Mobile edge computing (MEC)
• mobile cloud computing (MCC)
• offloading
• resources allocation
• MDP
• Q-learning