QoE-Oriented Rate Adaptation for DASH With Enhanced Deep Q-Learning

Abstract

With the popularity of handheld devices, the development of wireless communication technology and the proliferation of multimedia resources, mobile video has become the main business in LTE networks with explosive traffic demands. How to improve the quality of experience (QoE) of mobile video in the dynamic and complex network environment has become a research focus. Dynamic adaptive streaming over HTTP technology introduces adaptive bitrate (ABR) requests at the client side to improve video QoE and various rate adaptation algorithms are also constantly proposed. In view of the limitations of the existing heuristic or learning-based ABR methods, we propose redirecting enhanced Deep Q-learning toward DASH video QoE (RDQ), a QoE-oriented rate adaptation framework based on enhanced deep Q-learning. First, we establish a chunkwise subjective QoE model and utilize it as the reward function in reinforcement learning so that the strategy can converge toward the direction of maximizing the subjective QoE score. Then, we apply several effective improvements of deep Q-learning to the RDQ agent’s neural network architecture and learning mechanism to achieve faster convergence and higher average reward than other learning-based methods. The proposed RDQ agent has been thoroughly evaluated using trace-based simulation on the real-time LTE network data. For disparate network scenarios and different video contents, the RDQ agent can outperform the existing methods in terms of the QoE score. The breakdown analysis shows that RDQ can suppress the number and the duration of the stalling events to the minimum while maintaining high video bitrate, thus achieving better QoE performance than other methods.

Keywords

• Quality of experience (QoE)
• dynamic adaptive streaming over HTTP (DASH)
• enhanced deep-Q learning
• rate adaptation