Neural Adaptive Video Streaming with Pensieve

By Hongzi Mao, Ravi Netravali, Mohammad Alizadeh. Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, 2017

1,450 citations recorded by the source corpus.

Client-side video players employ adaptive bitrate (ABR) algorithms to optimize user quality of experience (QoE). Despite the abundance of recently proposed schemes, state-of-the-art ABR algorithms suffer from a key limitation: they use fixed control rules based on simplified or inaccurate models of the deployment environment. As a result, existing schemes inevitably fail to achieve optimal performance across a broad set of network conditions and QoE objectives. We propose Pensieve, a system that generates ABR algorithms using reinforcement learning (RL). Pensieve trains a neural network model that selects bitrates for future video chunks based on observations collected by client video players. Pensieve does not rely on pre-programmed models or assumptions about the environment. Instead, it learns to make ABR decisions solely through observations of the resulting performance of past decisions. As a result, Pensieve automatically learns ABR algorithms that adapt to a wide range of environments and QoE metrics. We compare Pensieve to state-of-the-art ABR algorithms using trace-driven and real world experiments spanning a wide variety of network conditions, QoE metrics, and video properties. In all considered scenarios, Pensieve outperforms the best state-of-the-art scheme, with improvements in average QoE of 12%--25%. Pensieve also generalizes well, outperforming existing schemes even on networks for which it was not explicitly trained.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/07d639436125ee204202cb0b34a081cde31e5214

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Learning to Schedule Communication in Multi-agent Reinforcement Learning (2019, background): “Playing the game of Go (Mnih et al., 2015), robotics control (Gu et al., 2017; Lillicrap et al., 2015), and adaptive video streaming (Mao et al., 2017) constitute just a few of the vast range of RL applications.”
Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization (2020, background): “Previous works have proposed controllers with different scalarized objective functions [43], but in many cases, engineers may prefer to learn the set of optimal trade-offs between their metrics of interest, rather than specify a scalarized objective in advance.”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, methodology): “In [65] and [67], the input of the DQL, i.”
Learning scheduling algorithms for data processing clusters (2018, methodology): “We use standard policy gradient methods that have recently been applied to multiple domains [29; 36; 40] for our learning algorithm.”
Learning in situ: a randomized experiment in video streaming (2019, methodology): “Each of the ABR algorithms we deployed has been evaluated in emulation in prior work [25, 46].”
Reinforcement Learning Based Scheduling Algorithm for Optimizing Age of Information in Ultra Reliable Low Latency Networks (2018, methodology): “Moreover, we used real bandwidth traces [11], [12], [13] to train our RL agent, and we set the packet drop probability to be 10%.”
A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP (2019, methodology): “Pensieve [102]3 is a framework that is built based on observations collected by DASH clients (i.e., throughput estimation and buffer occupancy) across large video streaming experiments.”
Flare: Practical Viewport-Adaptive 360-Degree Video Streaming for Mobile Devices (2018, methodology): “In non-360° video streaming, numerous rate adaptation schemes have been proposed [34, 36, 44, 59].”
Oboe: auto-tuning video ABR algorithms to network conditions (2018, methodology): “In §4, we discuss a recently proposed ABR algorithm based on a qualitatively different approach, reinforcement learning [39].”
A variegated look at 5G in the wild: performance, power, and QoE implications (2021, methodology): “Pensieve outperforms all other algorithms in 4G but incurs the highest video stall time under 5G setting.”
Edge Assisted Real-time Object Detection for Mobile Augmented Reality (2019, result): “Other video adaptation techniques [56–59] are also complementary to our work.”
Deep Learning in Mobile and Wireless Networking: A Survey (2018, methodology): “[311] Adaptive video bitrate Reinforcement learning A3C Oda et al.”