Feature-wise transformations

By Vincent Dumoulin, Ethan Perez, Nathan Schucher, Florian Strub, Harm de Vries, Aaron C. Courville, Yoshua Bengio. 2018

229 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Finding Generalizable Evidence by Learning to Convince Q&A Models (2019, methodology): “2405 into vectors and ; for each timestep’s features f from BERT, we scale and shift f element-wise: ( ⇤ f) + (Perez et al., 2018; Dumoulin et al., 2018).”
Analyzing and Improving the Image Quality of StyleGAN (2019, background): “Afﬁne transforms then produce styles that control the layers of the synthesis network g via adaptive instance normalization (AdaIN) [21, 9, 13, 8].”
A Style-Based Generator Architecture for Generative Adversarial Networks (2018, background): “Learned afﬁne transformations then specialize w to styles y = ( y s , y b ) that control adaptive instance normalization (AdaIN) [27, 17, 21, 16] operations after each convolution layer of the synthesis network g .”
WaveGrad: Estimating Gradients for Waveform Generation (2020, methodology): “The Feature-wise linear modulation (FiLM) (Dumoulin et al., 2018) module combines information from both noisy waveform and input mel-spectrogram.”
GAN Memory with No Forgetting (2020, background): “Given a d-dimensional feature h ∈ R from a layer of a neural network,3 FiLM yields ĥ = γ h+ β, (2) where ĥ is forwarded to the next layer, denotes the Hadamard product, and the scale γ ∈ R and shift β ∈ R may be conditioned on other information [18, 64, 17].”
StarGAN v2: Diverse Image Synthesis for Multiple Domains (2019, methodology): “s, we further enhance the training stability and construct a new baseline (C) by applying R 1 regularization [35] and switching the depth-wise concatenation to adaptive instance normalization (AdaIN) [9,15]. Note that we do not report LPIPS of these variations in Table1, since they are yet to be des”
Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion (2021, methodology): “We use conditional scaling (Dumoulin et al., 2018) to fuse the decoder hidden state h̃t with the embedding H to produce the time-dependent embeddings H̃: c̃ = Wth̃ H̃i = c̃ ·Hi,i={1,...,(m+n+|O”
Slimmable Neural Networks (2018, background): “Feature-wise transformation (Dumoulin et al., 2018) is a prevalent approach to integrate different sources of information, where conditional scales and biases are applied across the network.”
WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis (2021, methodology): “FiLM is the Feature-wise Linear Modulation [32] which combines information from yn and x.”
Learning a Universal Template for Few-shot Dataset Generalization (2021, methodology): “To this end, we propose to train a feature extractor jointly across diverse datasets using FiLM (Perez et al., 2018) in the form of conditional batch normalization: we share the parameters of the convolutional layers across datasets, but allocate a separate set of batch normalization parameters for…”
Controllable Medical Image Generation via Generative Adversarial Networks (2021, methodology): “In this paper, we adopt the structure from StyleGAN [14], where the latent space Z is ﬁrst mapped into the W space through a non-linear mapping network (an 8-layer MLP) and then merged into the synthesis network via adaptive instance normalization (AdaIN) at each convolutional layer [15, 16, 17].”
Deep Multimodal Fusion by Channel Exchanging (2020, background): “There is also a portion of the multimodal learning literature based on modulation [10, 12, 45].”
Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators (2019, background): “Finding an eﬀective way to represent the conditioning input is still an open question, and diﬀerent approaches have been proposed, such as feature-wise transformations [8].”
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation (2020, background): “(MHU-Net) [51]), conditioning strategies (either feature-wise linear modulation (FiLM) [52] or multiplicative conditioning), places of conditioning (at the bottleneck, at all the encoder layers or at the final decoder layer), context vectors (static visual context vector, visual-motion context vecto”
Side-Tuning: Network Adaptation via Additive Side Networks (2019, background): “Many problems are easier to solve when using a suite of sensors providing complementary sources of information [5].”
Contextual Transformation Networks for Online Continual Learning (2021, methodology): “Therefore, CNAPs are principally differs from CTN in that CNAPs assume having access to a well-pretrained knowledge source and uses FiLM to quickly adapt this knowledge to a new task.”
Deep Network Interpolation for Continuous Imagery Effect Transition (2018, background): “crease the model’s ﬂexibility. Concatenating extra conditions to inputs [46] or to middle features [22] alters the network behavior in various scenes. Modulating features with an afﬁne transformation [29,5,39] is able to effectively incorporate other information. Different from these works, we make ”
Adaptive Cross-Modal Few-Shot Learning (2019, methodology): “Task-dependent adaptive metric (TADAM) [35] relies on conditional batch normalization [5] to provide task adaptation (based on task representations encoded by visual features) to learn a task-dependent metric space.”