Deep Back-Projection Networks for Super-Resolution

By Muhammad Haris, Gregory Shakhnarovich, Norimichi Ukita. 2018

1,489 citations recorded by the source corpus.

The feed-forward architectures of recently proposed deep super-resolution networks learn representations of low-resolution inputs, and the non-linear mapping from those to high-resolution output. However, this approach does not fully address the mutual dependencies of low- and high-resolution images. We propose Deep Back-Projection Networks (DBPN), that exploit iterative up- and downsampling layers, providing an error feedback mechanism for projection errors at each stage. We construct mutually-connected up- and down-sampling stages each of which represents different types of image degradation and high-resolution components. We show that extending this idea to allow concatenation of features across up- and downsampling stages (Dense DBPN) allows us to reconstruct further improve super-resolution, yielding superior results and in particular establishing new state of the art results for large scaling factors such as 8Ã— across multiple data sets.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Hierarchical Neural Architecture Search for Single Image Super-Resolution (2020, methodology): “For the normal cell, we consider seven candidate operations, including identity mapping, 3× 3 and 5× 5 dilated convolution, 3× 3 and 5× 5 separable convolution, up and downprojection block (UDPB) [22], and residual channel attention block (RCAB) [8].”
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks (2018, background): “Various network architecture designs and training strategies have continuously improved the SR performance, especially the Peak Signal-to-Noise Ratio (PSNR) value [5,6,7,1,8,9,10,11,12].”
Image Super-Resolution Using Very Deep Residual Channel Attention Networks (2018, methodology): “We compare our method with 11 state-of-the-art methods: SRCNN [1], FSRCNN [2], SCN [3], VDSR [4], LapSRN [6], MemNet [9], EDSR [10], SRMDNF [11], D-DBPN [16], and RDN [17].”
Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search (2019, background): “Most of the hottest research focuses on non-linear mapping process (Dong et al., 2014; Lim et al., 2017; Ahn et al., 2018; Haris et al., 2018) where the ﬁnal performance matters most.”
Second-Order Attention Network for Single Image Super-Resolution (2019, methodology): “Speciﬁcally, we 1 (a) HR (b) FSRCNN (c) LapSRN [14] (d) SRMD [36] (e) EDSR [36] (f) DBPN [20] (g) RDN [6] (h) Ours Figure 1.”
TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution (2018, background): “[17], TOFlow [32], DBPN [6], RND [35], RCAN [34], SPMC [27], FSRVSR [22], and DUF [9].”
Dynamic Scene Deblurring With Parameter Selective Sharing and Nested Skip Connections (2019, methodology): “Further, skip connections were also used between the internal layers [34, 6, 38] to fully utilize different levels of features.”
Residual Dense Network for Image Restoration (2018, methodology): “Here, we also additionally use Flickr2K [33] as training data, which is also used in SRMDNF [14], and DDBPN [34].”
Unfolding the Alternating Optimization for Blind Super Resolution (2020, methodology): “From the arising of SRCNN [9], various DNNs [21, 40, 39, 10, 16, 18] have been proposed based on this setting.”
Feedback Network for Image Super-Resolution (2019, methodology): “[11] designed up- and down-projection units to achieve iterative error feedback.”
Unsupervised Learning for Real-World Super-Resolution (2019, background): “While traditionally addressed with classical techniques [19, 12, 29, 36, 15], current approaches [9, 10, 22, 23, 26, 11, 2, 3, 14, 16] employ deep learning methodologies to train a mapping from LR to HR.”
Geometric Back-Projection Network for Point Cloud Classification (2019, background): “ual tasks for relevant results. Some early work manages to involve the idea of correcting the error with feedback connections in CNNs: e.g., 2D human pose estimation [30], image super-resolution (SR) [31], [32]. In 3D, the recent PU-GAN [33] proposed an error feedback unit achieving feature expansio”
Multi-Scale Boosted Dehazing Network With Dense Feature Fusion (2020, methodology): “Motivated by the recent deep back-projection network for super-resolution [22], we adopt the strided convolutional/deconvolutional layers to learn the downsampling/upsampling operators in an end-to-end manner.”
Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data (2021, background): “The image super-resolution field [20, 23, 43, 16, 24, 26, 56, 21, 42, 55, 7, 29] has witnessed a variety of developments since SRCNN [8, 9].”
Closed-Loop Matters: Dual Regression Networks for Single Image Super-Resolution (2020, methodology): “Given paired training data, we follow the learning scheme of supervised SR methods [16, 26] and train model by minimizing Eqn.”
Interpreting Super-Resolution Networks with Local Attribution Maps (2020, methodology): “For networks from the literature, we select: SRCNN [14], FSRCNN, EDSR, LapSRN [29], SRResNet [30], DDBPN [21], RDN [61], MSRN Model Recpt.”
The 2018 PIRM Challenge on Perceptual Image Super-Resolution (2018, methodology): “The past few years have seen a major performance leap in single-image super-resolution (SR), both in terms of reconstruction accuracy (as measured e.g., by PSNR, SSIM) [19, 11, 39, 36, 38] and in terms of visual quality (as rated by human observers) [18, 24, 31, 42, 44].”
SRFlow: Learning the Super-Resolution Space with Normalizing Flow (2020, neutral): “This was addressed in URDGN [53], SRGAN [22] and more recent approaches [2,15,38,46] by adopting a conditional GAN based architecture and training strategy.”
Correction to: Single Image Super-Resolution via a Holistic Attention Network (2020, methodology): “We compare the proposed algorithm with 11 state-of-the-art methods: SRCNN [3], FSRCNN [4], VDSR [15], LapSRN [19], MemNet [31], SRMDNF [38], D-DBPN [6], RDN [41], EDSR [22], SRFBN [21] and SAN [2].”