Is the deconvolution layer the same as a convolutional layer?

By Wenzhe Shi, Jose Caballero, Lucas Theis, Ferenc Huszár, Andrew P. Aitken, Christian Ledig, Zehan Wang. 2016

148 citations recorded by the source corpus.

In this note, we want to focus on aspects related to two questions most people asked us at CVPR about the network we presented. Firstly, What is the relationship between our proposed layer and the deconvolution layer? And secondly, why are convolutions in low-resolution (LR) space a better choice? These are key questions we tried to answer in the paper, but we were not able to go into as much depth and clarity as we would have liked in the space allowance. To better answer these questions in this note, we first discuss the relationships between the deconvolution layer in the forms of the transposed convolution layer, the sub-pixel convolutional layer and our efficient sub-pixel convolutional layer. We will refer to our efficient sub-pixel convolutional layer as a convolutional layer in LR space to distinguish it from the common sub-pixel convolutional layer. We will then show that for a fixed computational budget and complexity, a network with convolutions exclusively in LR space has more representation power at the same speed than a network that first upsamples the input in high resolution space.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations (2017, methodology): “× In contrast to the “subpixel” layers [26, 27] used in [30], we use standard deconvolutions for simplicity.”
Deep Learning for Single Image Super-Resolution: A Brief Review (2018, methodology): “[55] further shows that when the ordinary but redundant nearest neighbor interpolation is replaced with the interpolation that pads the subpixels with zeroes, the deconvolution layer can be simplified into the subpixel convolution in ESPCN.”
Soft-to-Hard Vector Quantization for End-to-End Learned Compression of Images and Neural Networks (2017, methodology): “In contrast to the “subpixel” layers (Shi et al., 2016a;b) used in (Theis et al., 2017), we use standard deconvolutions for simplicity.”
ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content (2018, background): “As also mentioned in the previous section, it is argued [ODO16] that upsampling, especially the frequently used deconvolutional layers [SCT ∗ 16], cause checkerboard arte-facts.”
A Deep Journey into Super-resolution (2019, methodology): “The sub-pixel convolution operation used in this work is essentially similar to convolution transpose or deconvolution operation [76], where a fractional kernel stride is used to increase the spatial resolution of input feature maps.”