FVC: A New Framework towards Deep Video Compression in Feature Space
By Zhihao Hu, Guo Lu, Dong Xu. Computer Vision and Pattern Recognition, 2021
333 citations recorded by the source corpus.
Learning based video compression attracts increasing attention in the past few years. The previous hybrid coding approaches rely on pixel space operations to reduce spatial and temporal redundancy, which may suffer from inaccurate motion estimation or less effective motion compensation. In this work, we propose a feature-space video coding network (FVC) by performing all major operations (i.e., motion estimation, motion compression, motion compensation and residual compression) in the feature space. Specifically, in the proposed deformable compensation module, we first apply motion estimation in the feature space to produce motion information (i.e., the offset maps), which will be compressed by using the auto-encoder style network. Then we perform motion compensation by using deformable convolution and generate the predicted feature. After that, we compress the residual feature between the feature from the current frame and the predicted feature from our deformable compensation module. For better frame reconstruction, the reference features from multiple previous reconstructed frames are also fused by using the nonlocal attention mechanism in the multi-frame feature fusion module. Comprehensive experimental results demonstrate that the proposed framework achieves the state-of-the-art performance on four benchmark datasets including HEVC, UVG, VTL and MCL-JCV.
Subfields: Computer Vision and Pattern Recognition, Signal Processing
View Paper: https://www.semanticscholar.org/paper/adf34b98727b0f17604705d8a3b988daef0776ef
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
In Signal Processing
Citations in the corpus
- Neural Video Compression with Diverse Contexts (2023, methodology): “So recently it is also applied into NVC [23].”
- Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications (2022, positive): “DNN aided compression algorithms have achieved significant results, often outperforming state-of-the-art standardised codecs in a variety of source domains, from image [155], [156], [157], [158], [159], to video [160], [161], [162], [163], [164], [165], [166], [167], [168], speech [169], [170], and ”