Progressive Fusion Video Super-Resolution Network via Exploiting Non-Local Spatio-Temporal Correlations

By Peng Yi, Zhongyuan Wang, Kui Jiang, Junjun Jiang, Jiayi Ma. IEEE International Conference on Computer Vision, 2019

323 citations recorded by the source corpus.

Most previous fusion strategies either fail to fully utilize temporal information or cost too much time, and how to effectively fuse temporal information from consecutive frames plays an important role in video super-resolution (SR). In this study, we propose a novel progressive fusion network for video SR, which is designed to make better use of spatio-temporal information and is proved to be more efficient and effective than the existing direct fusion, slow fusion or 3D convolution strategies. Under this progressive fusion framework, we further introduce an improved non-local operation to avoid the complex motion estimation and motion compensation (ME&MC) procedures as in previous video SR approaches. Extensive experiments on public datasets demonstrate that our method surpasses state-of-the-art with 0.96 dB in average, and runs about 3 times faster, while requires only about half of the parameters.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/755227d7ea41b1d90df31f411f9ecbc998c20d26

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

VRT: A Video Restoration Transformer (2022, methodology): “I, we compare VRT with the state-of-the-art image and video SR methods [2], [8], [9], [10], [11], [12], [13], [14], [17], [21], [25], [31], [34].”
BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond (2020, methodology): “Our experiments suggest the use of bidirectional propagation scheme to maximize information gathering, and an optical ﬂow-based method to estimate the correspondence between two neighboring frames for feature alignment.”
Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring (2020, background): “Although self attention is implemented in recent video super-resolution work [25], to reduce memory requirement it resorts to pixelshuffling.”
PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization (2021, background): “78], [3, 15], [3, 15], and [50, 100], respectively.”
BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment (2021, methodology): “BI degradation BD degradation Params (M) Runtime (ms) REDS4 [21] Vimeo-90K-T [35] Vid4 [19] UDM10 [37] Vimeo-90K-T [35] Vid4 [19] Bicubic - 26.”