DVC: An End-To-End Deep Video Compression Framework

By Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Chunlei Cai, Zhiyong Gao. Computer Vision and Pattern Recognition, 2018

847 citations recorded by the source corpus.

Conventional video compression approaches use the predictive coding architecture and encode the corresponding motion information and residual information. In this paper, taking advantage of both classical architecture in the conventional video compression method and the powerful non-linear representation ability of neural networks, we propose the first end-to-end video compression deep model that jointly optimizes all the components for video compression. Specifically, learning based optical flow estimation is utilized to obtain the motion information and reconstruct the current frames. Then we employ two auto-encoder style neural networks to compress the corresponding motion and residual information. All the modules are jointly learned through a single loss function, in which they collaborate with each other by considering the trade-off between reducing the number of compression bits and improving quality of the decoded video. Experimental results show that the proposed approach can outperform the widely used video coding standard H.264 in terms of PSNR and be even on par with the latest standard H.265 in terms of MS-SSIM. Code is released at https://github.com/GuoLusjtu/DVC.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/ab995c4273111cde3e31ff7347c475aace10f1a5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Video Compression With Rate-Distortion Autoencoders (2019, background): “265, as well as the recent learned compression methods presented by [27] and [40].”
NeRV: Neural Representations for Videos (2021, background): “…representation are the output space and architecture Explicit (frame-based) Implicit (uniﬁed) Hand-crafted ( e.g ., HEVC [2]) Learning-based( e.g ., DVC [3]) Pixel-wise ( e.g ., NeRF [4] Image-wise(Ours) We also demonstrate the ﬂexibility of NeRV by exploring several applications it affords.”
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications (2022, positive): “DNN aided compression algorithms have achieved significant results, often outperforming state-of-the-art standardised codecs in a variety of source domains, from image [155], [156], [157], [158], [159], to video [160], [161], [162], [163], [164], [165], [166], [167], [168], speech [169], [170], and ”
Deep Contextual Video Compression (2021, background): “In addition, its predecessor DVC[3] is also tested.”
Artificial intelligence in the creative industries: a review (2020, methodology): “They have been applied to optimise a range of coding tools including intra prediction [219, 222], motion estimation [297], transforms [221], quantization [298], entropy coding [220] and loop filtering [299].”
Real-Time Intermediate Flow Estimation for Video Frame Interpolation (2020, background): “Optical ﬂow estimation is a long-standing vision task that aims to estimate the per-pixel motion, useful in many downstream tasks [55,65,33,64].”
Neural Video Compression with Diverse Contexts (2023, methodology): “The low delay encoding setting is used, as the same with most existing works [1, 28, 34].”
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing (2020, neutral): “A number of recent works [3, 16, 26, 43, 52, 65, 87] propose using a deep network to compress arbitrary videos.”
FVC: A New Framework towards Deep Video Compression in Feature Space (2021, methodology): “Most of the existing approaches [22, 19, 14] have to estimate the pixel-level optical flow maps and compress the corresponding pixel-level residual information.”
Scale-Space Flow for End-to-End Optimized Video Compression (2020, methodology): “264 in terms of PSNR and that outperforms HEVC in terms of MS-SSIM [24, 30].”
Finite Scalar Quantization: VQ-VAE Made Simple (2023, background): “Neural compression Many works (Ball´e et al., 2016; Minnen et al., 2018; Lu et al., 2019; Mentzer et al., 2020; Cheng et al., 2020) rely on unbounded scalar quantization and constrain the entropy of the quantized representation to prevent spreading to all integers.”