A guide to convolution arithmetic for deep learning

By Vincent Dumoulin, Francesco Visin. 2016

1,697 citations recorded by the source corpus.

We introduce a guide to help deep learning practitioners understand and manipulate convolutional neural network architectures. The guide clarifies the relationship between various properties (input shape, kernel shape, zero padding, strides and output shape) of convolutional, pooling and transposed convolutional layers, as well as the relationship between convolutional and transposed convolutional layers. Relationships are derived for various cases, and are illustrated in order to make them intuitive.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Adversarially Learned Inference (2016, background): “For more details about transposed convolutions, see Dumoulin and Visin (2016).”
Generative and Discriminative Voxel Modeling with Convolutional Neural Networks (2016, neutral): “Upsampling in the decoder network is accomplished via fractionally strided convolutions, implemented as the gradient of an equivalent strided convolution[10], in every second layer.”
UWSpeech: Speech to Speech Translation for Unwritten Languages (2020, background): “As shown in Figure 2, the inverter leverages several transposed convolution layers [8] to increase the length of e z by c × (opposed to the 1 /c × in the converter), to match the length of the original mel-spectrogram sequence.”
Demystifying Parallel and Distributed Deep Learning (2018, background): “Note that the formula omits various extensions of the operator [71], such as variable stride, padding, and dilation [262], each of which modifies the accessed indices and H ′,W ′.”
Fast Spectrogram Inversion Using Multi-Head Convolutional Neural Networks (2018, neutral): “Based on these motivations, we propose the multi-head convolutional neural network (MCNN) architecture, where each head is composed of L transposed convolution layers (please see [14] for detailed explanations and formulations of”
Fast Generation for Convolutional Autoregressive Models (2017, methodology): “We show an example model with 2 convolutional and 2 transposed convolutional layers each with a stride of 2 (Dumoulin & Visin, 2016).”
TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech Separation (2017, background): “For each segment, this operation can also be formulated as a linear deconvolutional operation (also known as transposed convolution) [24], where each row in B corresponds to a 1-D filter which is jointly learned together with the other parts of the network.”
Improving MMD-GAN Training with Repulsive Loss Function (2018, background): “Consider v′ = W Tdbcu = [∂u∂v ] Tu which is essentially the transpose convolution of Wc on u (Dumoulin & Visin (2016)).”
Inductive biases for deep learning of higher-level cognition (2020, background): “…Bishop, 1995; Srivastava et al., 2014; Kukačka et al., 2017; Zhang et al., 2017), architectural constraints (Yu and Koltun, 2015; Long et al., 2015; Dumoulin and Visin, 2016; He et al., 2016; Huang et al., 2017), parameter sharing (Hochreiter and Schmidhuber, 1997; Pham et al., 2018), implicit…”
Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation (2018, background): “Transposed convolutions with a ﬁlter size of k and a stride of x > 1 can be viewed as convolutions applied to feature maps padded with x − 1 zeros between each original value [2].”
In silico labeling: Predicting fluorescent labels in unlabeled images (2018, methodology): “For the in-scale conﬁguration, we trim off a size 1 border in the row and column dimensions, corresponding to a valid (non zero-padded) convolution with a kernel size of 3 and a stride of 1 (Dumoulin and Visin, 2016).”
INet: Convolutional Networks for Biomedical Image Segmentation (2021, background): “The Conv changes theoretical receptive ﬁelds (TRF) by the process as follows [31]: where r i and r i − 1 denote the sizes of the receptive ﬁeld for the output features of the i th and ( i − 1) th Convs, respectively; j i − 1 and j i represent the distance of two adjacent features in the output…”
Deep Count: Fruit Counting Based on Deep Simulated Learning (2017, result): “We compared our results with several methods, namely area-based technique, shallow neural network, and our network with the original Inception-ResNet.”
An Exploratory Analysis of Multi-Class Uncertainty Approximation in Bayesian Convolutional Neural Networks (2018, background): “See [Dumoulin and Visin, 2016] for an exhaustive overview of the arithmetics of convolutional layers.”