Joint Content Caching, Recommendation, and Transmission for Layered Scalable Videos Over Dynamic Cellular Networks: A Dueling Deep Q-Learning Approach

Abstract

Scalable Video Coding (SVC) and edge caching are two techniques that hold the potential to improve user-perceived video viewing experience. Moreover, video recommendation can further enhance the caching gain by reshaping users’ video preferences. In this paper, we investigate the video caching, recommendation and transmission for layered SVC streaming in cache-enabled cellular networks. Considering the dynamic characteristics of video popularity distribution and wireless network environment, to improve energy efficiency by minimizing system energy consumption and ensure the average user preference deviation tolerance, we begin by formulating a long-term optimization problem that focuses on video caching, recommendation and user association (UA). The problem is then transformed into a Markov decision process (MDP), which is solved by designing a dueling deep Q-learning network (DDQN)-based algorithm. Using this algorithm, we can obtain the optimal video caching, recommendation and UA solutions. Since the action space of the MDP is huge, to cope with the “curse of dimensionality”, linear approximation is integrated into the designed algorithm. Finally, the proposed algorithm’s convergence and effectiveness in reducing long-term system energy consumption are demonstrated through extensive simulations.

Keywords

• Scalable video coding
• edge caching
• recommendation
• user association
• energy efficiency
• dueling deep Q-learning