Deep Set Prediction Networks

By Yan Zhang, Jonathon S. Hare, Adam Prügel-Bennett. 2019

130 citations recorded by the source corpus.

Current approaches for predicting sets from feature vectors ignore the unordered nature of sets and suffer from discontinuity issues as a result. We propose a general model for predicting sets that properly respects the structure of sets and avoids this problem. With a single feature vector as input, we show that our model is able to auto-encode point sets, predict the set of bounding boxes of objects in an image, and predict the set of attributes of these objects.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Object-Centric Learning with Slot Attention (2020, methodology): “Neural networks for sets A range of recent methods explore set encoding [34, 50, 51], generation [31, 52], and set-to-set mappings [20, 53].”
Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation (2021, methodology): “We identify and show that deep set prediction networks (DSPN) (Zhang et al., 2019; Huang et al., 2020) satisfy this property with a speciﬁc choice of set encoder.”
Set-Structured Latent Representations (2020, methodology): “The second is producing a set-like structure as output; this is standard in object detection or neural set generation (Zhang et al., 2019; Vinyals et al., 2015; Rezatofighi et al., 2018).”
Conditional Structure Generation through Graph Variational Generative Adversarial Nets (2019, methodology): “Finally, the idea of enabling permutation-invariance in an encoder-decoder system also applies to problems like set generation, where we notice the recent development of models that achieve it by backpropagating through an order-invariant set encoder [49], which can be further improved with more sop”
Conditional Set Generation with Transformers (2020, methodology): “Following Zhang et al. (2019), we use the CLEVR dataset to test the efﬁcacy of our models for object detection in a simple setting, which might pave the way for more advanced object-based inference in future works.”
Entity Abstraction in Visual Model-Based Reinforcement Learning (2019, background): “Whereas a neural network encoder is often used to approximate the posterior [1, 3, 4], a single forward pass that computes q in parallel for each entity is insufficient to break the symmetry for dividing responsibility of modeling different objects among the entity variables [53] because the entitie”
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations (2020, methodology): “Deep Set Prediction Networks [58], Transformer Set Prediction Networks [20] or Mask-RCNN based models [11] are viable alternatives to the Slot Attention module as concept embedding module.”
multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning (2021, methodology): “Thus, we use a set-based Hungarian Loss (Zhang et al., 2019a,b) for capturing the permutation-invariant nature of generating a set of proofs.”
Set Prediction without Imposing Structure as Conditional Density Estimation (2020, methodology): “Similarly, deep set prediction network (DSPN) (Zhang et al., 2019) applies the bi-level optimization scheme (Domke, 2012) for learning.”
Better Set Representations For Relational Reasoning (2020, methodology): “For instance, deep set prediction networks [23] require an additional loss term that uses the ground truth sets during training, which makes them unsuitable for our setting where ground truth sets are not available.”
Learn to Predict Sets Using Feed-Forward Neural Networks (2020, background): “Somewhat surprisingly, there are only few works on learning to predict sets using deep neural networks [8], [9], [13].”