A Pooling Approach to Modelling Spatial Relations for Image Retrieval and Annotation

By Mateusz Malinowski, Mario Fritz. 2014

29 citations recorded by the source corpus.

Over the last two decades we have witnessed strong progress on modeling visual object classes, scenes and attributes that have significantly contributed to automated image understanding. On the other hand, surprisingly little progress has been made on incorporating a spatial representation and reasoning in the inference process. In this work, we propose a pooling interpretation of spatial relations and show how it improves image retrieval and annotations tasks involving spatial language. Due to the complexity of the spatial language, we argue for a learning-based approach that acquires a representation of spatial relations by learning parameters of the pooling operator. We show improvements on previous work on two datasets and two different tasks as well as provide additional insights on a new dataset with an explicit focus on spatial relations.

Subfields: Computer Vision and Pattern Recognition, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

Citations in the corpus

A simple neural network module for relational reasoning (2017, background): “d on either rule-based spatial representations or hand-engineered spatial features [8, 10, 20, 21, 24, 29, 38, 39]. Although there are some attempts to learn spatial relations using spatial templates [28, 30], these approaches are less versatile than ours. Visual question answering Visual question a”
Learning Like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images (2015, background): “Very recent works of image captioning includes [39, 25, 24, 55, 10, 15, 7, 32, 37, 26, 57, 35].”