Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections

By Xiao-Jiao Mao, Chunhua Shen, Yubin Yang. 2016

429 citations recorded by the source corpus.

Image restoration, including image denoising, super resolution, inpainting, and so on, is a well-studied problem in computer vision and image processing, as well as a test bed for low-level image modeling algorithms. In this work, we propose a very deep fully convolutional auto-encoder network for image restoration, which is a encoding-decoding framework with symmetric convolutional-deconvolutional layers. In other words, the network is composed of multiple layers of convolution and de-convolution operators, learning end-to-end mappings from corrupted images to the original ones. The convolutional layers capture the abstraction of image contents while eliminating corruptions. Deconvolutional layers have the capability to upsample the feature maps and recover the image details. To deal with the problem that deeper networks tend to be more difficult to train, we propose to symmetrically link convolutional and deconvolutional layers with skip-layer connections, with which the training converges much faster and attains better results.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Generative Models

Citations in the corpus

Progressive Growing of GANs for Improved Quality, Stability, and Variation (2017, methodology): “…recently that formulation has been improved (Hjelm et al., 2017) and a number of more stable alternatives have been proposed, including least squares (Mao et al., 2016b), absolute deviation with margin (Zhao et al., 2017), and Wasserstein distance (Arjovsky et al., 2017; Gulrajani et al., 2017).”
WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images (2017, methodology): “Inspired by recent work on image restoration and denoising using neural networks [20][21], we incorporate skipping layers on the basic encoder-decoder structure to compensate for the loss in high frequency components through the network.”
Noise Reduction in ECG Signals Using Fully Convolutional Denoising Autoencoders (2019, background): “Contrary to convolutional layers which connect multiple input activations to a single activation, deconvolutional layers project a single input activation into multiple outputs [31].”
From Motion Blur to Motion Flow: A Deep Learning Solution for Removing Heterogeneous Motion Blur (2016, methodology): “Some end-to-end methods [17, 25] are proposed to reconstruct blur-free images, however, they can only handle mild Gaussian blur.”
Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder (2017, methodology): “We know from earlier image restoration results (e.g. [Mao et al. 2016]) that our training set should contain at least hundreds of image sequences that o er considerable variety if we want the trained network to work well in all scenes.”
Noisier2Noise: Learning to Denoise From Unpaired Noisy Data (2019, neutral): “Convolutional neural networks have proven to be powerful tools for many image processing tasks, including denoising[13], inpainting[18], and domain translation[5].”
Noise2Noise: Learning Image Restoration without Clean Data (2018, methodology): “Our baseline is a recent state-of-the-art method ”RED30” (Mao et al., 2016), a 30-layer hierarchical residual network with 128 feature maps, which has been demonstrated to be very effective in a wide range of image restoration tasks, including Gaussian noise.”
Content-aware image restoration: pushing the limits of fluorescence microscopy (2018, methodology): “1The network architecture we used is based on [31, 32] (Supp.”
Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk (2017, background): “These more sophisticated priors are recently emerging in empirical applications of many fields of imaging, such as medical imaging [19]–[32], microscopy [33], inpainting [34], [35], superresolution [36]–[38], compressed sensing [39]–[42], image manipulation [43], and many more.”
Image Super-Resolution Using Dense Skip Connections (2017, background): “Many symmetric skip connections were introduced in an encoding-decoding network [17] for image restoration tasks.”
Long-Term Image Boundary Prediction (2016, methodology): “Resolution is maintained by skip connections, as in (Mao, Shen, and Yang 2016).”
An Efficient Group Recommendation Model With Multiattention-Based Neural Networks (2020, methodology): “Based on the obtained samples, GLE trains a ﬁve-layer SDAE [44] to get an n de -dimensional descriptive feature vector for each group.”