ABRaider: Multiphase Reinforcement Learning for Environment-Adaptive Video Streaming

Abstract

HTTP-based video streaming technology is widely used in today’s video delivery services. The streaming solution uses the adaptive bitrate (ABR) algorithm for better video quality and user experience. Despite many efforts to improve the quality of experience (QoE), it is very challenging for ABR algorithms to guarantee high QoE to all users in various environments. The video streaming circumstances in the real world have become even more complicated by the proliferation of mobile devices, high-quality content, and heterogeneous configurations of video players. Many ABR algorithms aim to find monotonous strategies that generally perform well without focusing on the complexity of the environments, which can degrade performance. In this paper, we propose ABRaider that guarantees high QoE to all users in a variety of environments in the real world while being generalized with multiple strategies and specialized in each user’s environment. In ABRaider, we propose multi-phase RL consisting of offline and online phases. In the offline phase, ABRaider integrates the strengths of the ABR algorithms and develops policies suitable for various environments. In the online phase, ABRaider focuses on specializing in the environments of individual users by leveraging the computational power of the clients. Experiment results show that ABRaider outperforms existing solutions in various environments, achieving 19.9% and 42.2% QoE improvement in VoD and live streaming, respectively.

Keywords

• Adaptive bitrate algorithm
• federated learning
• quality of experience
• reinforcement learning
• video streaming