Are Elephants Bigger than Butterflies? Reasoning about Sizes of Objects

By Hessam Bagherinezhad, Hannaneh Hajishirzi, Yejin Choi, Ali Farhadi. 2016

52 citations recorded by the source corpus.

Human vision greatly benefits from the information about sizes of objects. The role of size in several visual reasoning tasks has been thoroughly explored in human perception and cognition. However, the impact of the information about sizes of objects is yet to be determined in AI. We postulate that this is mainly attributed to the lack of a comprehensive repository of size information. In this paper, we introduce a method to automatically infer object sizes, leveraging visual and textual information from web. By maximizing the joint likelihood of textual and visual observations, our method learns reliable relative size estimates, with no explicit human supervision. We introduce the relative size dataset and show that our method outperforms competitive textual and visual baselines in reasoning about size comparisons.

Subfields: Computer Vision and Pattern Recognition, Natural Language Processing, Knowledge Graphs & Representation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Knowledge Graphs & Representation

Citations in the corpus

Language Is Not All You Need: Aligning Perception with Language Models (2023, methodology): “…SST-2 [34] OCR-free sentiment classification Accuracy ✓ HatefulMemes [35] OCR-free meme classification ROC AUC ✓ Cross-modal transfer RelativeSize [36] Commonsense reasoning (object size) Accuracy ✓ MemoryColor [37] Commonsense reasoning (object color) Accuracy ✓ ColorTerms [38] Commonsense…”
Temporal Common Sense Acquisition with Minimal Supervision (2020, background): “These works include but are not limited to size, weight, and strength (Bagherinezhad et al., 2016; Forbes and Choi, 2017; Elazar et al., 2019), roundness and deli-ciousness (Yang et al., 2018), and intensity (Cocos et al., 2018).”
How Large Are Lions? Inducing Distributions over Quantitative Attributes (2019, unknown): “These approaches focus on facilitating comparisons between quantitative attributes of nouns (Bagherinezhad et al., 2016; Forbes and Choi, 2017; Yang et al., 2018), intensity of adjectives (De Melo and Bansal, 2013; Cocos et al., 2018) and coarse classiﬁcation of events duration and relative order…”
Visually-Augmented Language Modeling (2022, methodology): “Remarkably, our method substantially improves +14.50%, +17.80%, and +11.68% accuracy on M EMORY C OLOR , R ELATIVE S IZE and O BJECT S HAPE datasets, respectively.”
Verb Physics: Relative Physical Knowledge of Actions and Objects (2017, background): “Thus, our work provides unique value to complement recent attempts to acquire commonsense knowledge from web images (Izadinia et al., 2015; Bagherinezhad et al., 2016; Sadeghi et al., 2015).”
Is the Red Square Big? MALeViC: Modeling Adjectives Leveraging Visual Contexts (2019, background): “Bagherinezhad et al. (2016), for example, assess whether elephants are bigger than butterflies.”
Incorporating Structured Commonsense Knowledge in Story Completion (2018, background): “Related works include (Bagherinezhad et al. 2016), which builds a knowledge graph and uses it to deduce the size of objects (Bagherinezhad et al.”