Visual Reasoning by Progressive Module Networks

By Seung Wook Kim, Makarand Tapaswi, Sanja Fidler. 2018

14 citations recorded by the source corpus.

Humans learn to solve tasks of increasing complexity by building on top of previously acquired knowledge. Typically, there exists a natural progression in the tasks that we learn - most do not require completely independent solutions, but can be broken down into simpler subtasks. We propose to represent a solver for each task as a neural module that calls existing modules (solvers for simpler tasks) in a functional program-like manner. Lower modules are a black box to the calling module, and communicate only via a query and an output. Thus, a module for a new task learns to query existing modules and composes their outputs in order to produce its own output. Our model effectively combines previous skill-sets, does not suffer from forgetting, and is fully differentiable. We test our model in learning a set of visual reasoning tasks, and demonstrate improved performances in all tasks by learning progressively. By evaluating the reasoning process using human judges, we show that our model is more interpretable than an attention-based baseline.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Jorge Armando Mendez Mendez
  2. Eric Eaton
  3. Mohammad Rostami
  4. Oleksiy Ostapenko
  5. Laurent Charlin

In Computer Vision and Pattern Recognition

  1. Jorge Armando Mendez Mendez
  2. Eric Eaton

In Model Architectures

  1. Jorge Armando Mendez Mendez
  2. Eric Eaton

Citations in the corpus