Understanding and predicting importance in images

By Alexander C. Berg, Tamara L. Berg, Hal Daumé Iii, Jesse Dodge, Amit Goyal, Xufeng Han, Alyssa C. Mensch, Margaret Mitchell, Aneesh Sood, Karl Stratos and 1 more. 2012

164 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Understanding and Evaluating Racial Biases in Image Captioning (2021, methodology): “Distance is predictive because darker-skinned individuals tend to be further from the image center than lighter-skinned individuals; this is troubling since the “important” parts of an image tend to be more centered [11].”
CIDEr: Consensus-based image description evaluation (2014, background): “[2] characterize the relative importance of objects (nouns).”
Visual Madlibs: Fill in the blank Image Generation and Question Answering (2015, background): “Whether an object is mentioned in an image description can be viewed as an indication of the object’s importance [2].”
Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels (2015, methodology): “We train all these classifiers jointly and end-to-end as multiple “heads” branching from the same shared convolutional neural network (ConvNet) trunk [27, 46].”
Visual Relationship Detection with Language Priors (2016, background): “A meaning space of relationships have aided the cognitive task of mapping images to captions [35,36,37,38].”
Don't just listen, use your imagination: Leveraging visual common sense for non-visual tasks (2015, background): “Example tasks include reasoning about what people talk about in images [4], understanding the flow of time (when) [39], identifying where the image is taken [24, 28] and judging the intentions of people in images (why) [40].”
A Case Study on Combining ASR and Visual Features for Generating Instructional Video Captions (2019, methodology): “10This oracle is unlikely to be achievable, as it assumes 100% precision for the 62 objects considered (which also implies modeling which objects to talk about, a non-trivial task in itself (Berg et al., 2012)).”
Adopting Abstract Images for Semantic Scene Understanding (2016, background): “The order in which the attributes are likely to be named was studied in [38].”
How Do We Talk about Other People? Group (Un)Fairness in Natural Language Image Descriptions (2019, background): “Berg and colleagues emphasized that crowd annotations are “human-centric annotations,” which provide a wealth of information about how people judge images, and what they find important (Berg et al. 2012).”
It Takes Two to Tango: Towards Theory of AI's Mind (2017, background): “To the best of our knowledge, this is the first evaluation that measures the extent to which these modalities allow a human to build a mental model for the AI and predict its behavior.”
The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning (2022, background): “In images, [5] show that different object types have different likelihoods of being mentioned in image captions (e.”
Detecting Visual Relationships with Deep Relational Networks (2017, methodology): “They are essentially different from our work, which aims to provide a method dedicated to generic visual relationship detection.”
Learning Common Sense through Visual Abstraction (2015, background): “Recently, there has been a surge in interest in high-level “beyond recognition” tasks which can beneﬁt from external knowledge beyond what is depicted in the image [3, 19, 23, 30, 31].”
What's "up" with vision-language models? Investigating their struggle with spatial reasoning (2023, background): “See Berg et al. (2012) for a discussion of how descriptions manifest according to similar factors in crowdsourced image captioning corpora.”
Best of both worlds: Human-machine collaboration for object annotation (2015, background): “However, since some objects in the image may be more important than others [49, 25, 2], the requester may optionally specify a function mapping each image region and class label to a real value indicating importance.”
A Sentence Is Worth a Thousand Pixels (2013, background): “Moreover, natural language parsing algorithms are not perfect, resulting in noisy estimates.”
Nonparametric Method for Data-driven Image Captioning (2014, background): “com/CLSP11/ (3)In particular, papers stemming from the 2011 JHU-CLSP Summer Workshop (Berg et al., 2012; Dodge et al., 2012; Mitchell et al., 2012) and more recently, the best paper award winner at ICCV (Ordonez et al.”
Salient object detection: A survey (2014, background): “Some topics that are closely or remotely related to visual saliency include: salient object detection [41], fixation prediction [33, 34], object importance [17, 157, 182], memorability [78], scene clutter [169], video interestingness [50, 69, 90, 96], surprise [80], image quality assessment [208, 20”
Deep Classifiers from Image Tags in the Wild (2015, background): “, person) or they are not an important subject in the scene [2] (e.”