Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs

By Michael Gygli, Mohammad Norouzi, Anelia Angelova. 2017

68 citations recorded by the source corpus.

We approach structured output prediction by optimizing a deep value network (DVN) to precisely estimate the task loss on different output configurations for a given input. Once the model is trained, we perform inference by gradient descent on the continuous relaxations of the output variables to find outputs with promising scores from the value network. When applied to image segmentation, the value network takes an image and a segmentation mask as inputs and predicts a scalar estimating the intersection over union between the input and ground truth masks. For multi-label classification, the DVN's objective is to correctly predict the F1 score for any potential label configuration. The DVN framework achieves the state-of-the-art results on multi-label prediction and image segmentation benchmarks.

Subfields: Artificial Intelligence, Generative Models, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Generative Models

In Optimization & Training

Citations in the corpus

Structured Convolutional Kernel Networks for Airline Crew Scheduling (2021, methodology): “Furthermore, inspired by Q-learning, Gygli et al. (2017) used an oracle value function as the objective for energy-based deep networks, and Belanger & McCallum (2016) introduced structured prediction energy networks (SPENs) to address the inductive bias and to learn discriminative features of the st”
End-to-End Learning for Structured Prediction Energy Networks (2017, unknown): “In recent work, Gygli et al. (2017) propose an alternative learning method that ﬁts the energy function such that E x ( · ) ≈ − ∆( · , y ∗ ) , where ∆ is deﬁned as in Sec.”
Fast Task-Aware Architecture Inference (2019, methodology): “In particular, the proposed framework builds on top of the Deep Value Networks (DVNs) [8] that were originally developed in the context of structured prediction applied to image segmentation.”
Neural Message Passing for Multi-Label Classification (2019, methodology): “To overcome the naive pairwise dependency constraint of CRFs, structured prediction energy networks (SPENS) [5] and related methods [50, 20] locally optimize an unconstrained structured output.”
Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections (2019, methodology): “Others iteratively update the inputs of a model [23,25,29], e.”
Deep Structured Prediction with Nonlinear Output Transformations (2018, methodology): “[15] are another gradient based approach which uses the same architecture and relaxation as SPENs.”
Energy-Based Models for Continual Learning (2020, methodology): “EBMs are powerful models that have been applied to different domains, such as structured prediction (Belanger & McCallum, 2016; Gygli et al., 2017; Rooshenas et al., 2019; Tu & Gimpel, 2019), machine translation (Tu et al.”