Joint Learning of Set Cardinality and State Distribution

By Seyed Hamid Rezatofighi, Anton Milan, Javen Qinfeng Shi, Anthony R. Dick, Ian David Reid. 2017

16 citations recorded by the source corpus.

We present a novel approach for learning to predict sets using deep learning. In recent years, deep neural networks have shown remarkable results in computer vision, natural language processing and other related problems. Despite their success,traditional architectures suffer from a serious limitation in that they are built to deal with structured input and output data,i.e. vectors or matrices. Many real-world problems, however, are naturally described as sets, rather than vectors. Existing techniques that allow for sequential data, such as recurrent neural networks, typically heavily depend on the input and output order and do not guarantee a valid solution. Here, we derive in a principled way, a mathematical formulation for set prediction where the output is permutation invariant. In particular, our approach jointly learns both the cardinality and the state distribution of the target set. We demonstrate the validity of our method on the task of multi-label image classification and achieve a new state of the art on the PASCAL VOC and MS COCO datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ian David Reid
2. L. Leal-Taix'e
3. Anton Milan
4. Michal Drozdzal
5. Adriana Romero
6. Daniel Cremers
7. Amaia Salvador
8. Javen Qinfeng Shi

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. L. Leal-Taix'e
4. Ming-Hsuan Yang
5. Chunhua Shen
6. Ian David Reid
7. Bernt Schiele
8. Jitendra Malik

In Probabilistic & Bayesian Methods

1. Adriana Romero
2. Ian David Reid
3. Daniel Cremers
4. Javen Qinfeng Shi
5. Michal Drozdzal
6. Anton Milan

Citations in the corpus

• Elucidating image-to-set prediction: An analysis of models, losses and datasets (2019, methodology): “al. Figure4presents the test O-F1 metric and cardinalCOCO VOC Recipe1M Li et al. [24] 62.90 79.10 - Zhu et al. [61] 75.80 - - Liu et al. [29] 74.00 - - Liu et al. [38] 69.00 78.60 - Rezatofighi et al. [39] 70.70 81.50 - Rezatofighi et al. [57] 66.50 62.90 - Chen et al. [7] 71.10 - - Chen et al. [6] 67”
• Inverse Cooking: Recipe Generation From Food Images (2018, methodology): “Finally, we consider recently proposed models which couple set prediction with cardinality prediction to determine which elements to include in the set [44].”
• Learn to Predict Sets Using Feed-Forward Neural Networks (2020, background): “However, many real-world problems are naturally described as unstructured data such as sets [7], [8].”