BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond

By Kelvin C. K. Chan, Xintao Wang, Ke Yu, Chao Dong, Chen Change Loy. Computer Vision and Pattern Recognition, 2020

663 citations recorded by the source corpus.

Video super-resolution (VSR) approaches tend to have more components than the image counterparts as they need to exploit the additional temporal dimension. Complex designs are not uncommon. In this study, we wish to untangle the knots and reconsider some most essential components for VSR guided by four basic functionalities, i.e., Propagation, Alignment, Aggregation, and Upsampling. By reusing some existing components added with minimal redesigns, we show a succinct pipeline, BasicVSR, that achieves appealing improvements in terms of speed and restoration quality in comparison to many state-of-the-art algorithms. We conduct systematic analysis to explain how such gain can be obtained and discuss the pitfalls. We further show the extensibility of BasicVSR by presenting an information-refill mechanism and a coupled propagation scheme to facilitate information aggregation. The BasicVSR and its extension, IconVSR, can serve as strong baselines for future VSR approaches.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/084576d2b607dfe3796b2fd5db2de9eb541e9aa9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Low-Light Image and Video Enhancement Using Deep Learning: A Survey (2021, background): “and video super-resolution [105], low-light video enhancement receives less attention.”
FlowFormer: A Transformer Architecture for Optical Flow (2022, background): “In many downstream video tasks, such as action recognition [46,37,61], video inpainting [28,51,13], video super-resolution [30,5,39], and frame interpolation [52,33,20], optical flow serves as a fundamental component providing dense correspondences as valuable clues for prediction.”
VRT: A Video Restoration Transformer (2022, methodology): “2) Temporal Alignment and Fusion: Since supporting frames are often highly-related but misaligned, temporal alignment plays an critical role in video restoration [2], [3], [12], [13], [31].”
BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment (2021, methodology): “In particular, IconVSR (the more powerful variant of BasicVSR) [3] consists of a coupled propagation scheme with sequentially-connected branches to facilitate information exchange.”