FiLM: Visual Reasoning with a General Conditioning Layer

By Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, Aaron C. Courville. 2017

4,130 citations recorded by the source corpus.

We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation. FiLM layers influence neural network computation via a simple, feature-wise affine transformation based on conditioning information. We show that FiLM layers are highly effective for visual reasoning - answering image-related questions which require a multi-step, high-level process - a task which has proven difficult for standard deep learning methods that do not explicitly model reasoning. Specifically, we show on visual reasoning tasks that FiLM layers 1) halve state-of-the-art error for the CLEVR benchmark, 2) modulate features in a coherent manner, 3) are robust to ablations and architectural modifications, and 4) generalize well to challenging, new data from few examples or even zero-shot.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Yoshua Bengio
  3. Mohit Bansal
  4. Jianfeng Gao
  5. Quoc V. Le
  6. Percy Liang
  7. Trevor Darrell
  8. Ruslan Salakhutdinov

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Ming-Hsuan Yang
  5. Xiaogang Wang
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Bernt Schiele

In Multimodal Learning

  1. Zhe Gan
  2. Mohit Bansal
  3. Devi Parikh
  4. Trevor Darrell
  5. Marcus Rohrbach
  6. Ronghang Hu
  7. Linjie Li
  8. Dhruv Batra

Citations in the corpus