MuZero with Self-competition for Rate Control in VP9 Video Compression

By Amol Mandhane, A. Zhernov, Maribeth Rauh, Chenjie Gu, Miaosen Wang, Flora Xue, Wendy Shang, Derek Pang, Rene Claus, Ching-Han Chiang and 9 more. 2022

48 citations recorded by the source corpus.

Video streaming usage has seen a significant rise as entertainment, education, and business increasingly rely on online video. Optimizing video compression has the potential to increase access and quality of content to users, and reduce energy use and costs overall. In this paper, we present an application of the MuZero algorithm to the challenge of video compression. Specifically, we target the problem of learning a rate control policy to select the quantization parameters (QP) in the encoding process of libvpx, an open source VP9 video compression library widely used by popular video-on-demand (VOD) services. We treat this as a sequential decision making problem to maximize the video quality with an episodic constraint imposed by the target bitrate. Notably, we introduce a novel self-competition based reward mechanism to solve constrained RL with variable constraint satisfaction difficulty, which is challenging for existing constrained RL methods. We demonstrate that the MuZero-based rate control achieves an average 6.28% reduction in size of the compressed videos for the same delivered video quality level (measured as PSNR BD-rate) compared to libvpx's two-pass VBR rate control policy, while having better constraint satisfaction behavior.

Subfields: Electrical and Electronic Engineering, Management Science and Operations Research, Reinforcement Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

In Management Science and Operations Research

In Reinforcement Learning

In Signal Processing

Citations in the corpus

Adversarial Training for High-Stakes Reliability (2022, background): “Advances in deep learning have led to increasingly powerful AI systems, for example in sequential decision making [1, 2, 3, 4], robotics [5, 6], and language modeling and text-based reasoning [7, 8, 9, 10, 11].”
A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, methodology): “[177] leverage the Muzero [243], one of the alpha series algorithms, to solve the safe RL problem in video compression.”
A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, unknown): “…autonomous driving and robotics of safe RL applications, safe RL is also adopted to ensure safety in recommender systems [231], video compression [70], video transmissions [232], wireless security [233], satellite docking [234], edge computing [235], chemical processes [43] and vehicle schedule…”