Annotator rationales for visual recognition

By Jeff Donahue, Kristen Grauman. 2011

78 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Shaping Visual Representations with Language for Few-Shot Classification (2019, neutral): “In this framework, learning with attributes and other domain-speciﬁc rationales has been tackled extensively [8, 9, 29], but language remains relatively unexplored.”
Benefits of Intermediate Annotations in Reading Comprehension (2020, background): “In the ﬁeld of computer vision, Donahue and Grauman (2011) collected similar annotations, for visual recognition, where crowd-workers high-lighted relevant regions in images.”
Yin and Yang: Balancing and Answering Binary Visual Questions (2015, positive): “This signal can be used as additional supervision for training models such as [41, 11, 31, 6] that can leverage explanations provided by the annotator in addition to labels.”
Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations (2017, methodology): “For SVMs and speciﬁc text classiﬁcation architectures, there exists work on incorporating human input into decision boundaries in the form of annotator rationales [Zaidan et al. , 2007; Donahue and Grauman, 2011; Zhang et al. , 2016].”
Leveraging the Wisdom of the Crowd for Fine-Grained Recognition (2016, background): “These tasks yield a great deal of information for a human user and can thus add tremendous value to society.”
Information Bottleneck Learning Using Privileged Information for Visual Recognition (2016, background): “The same idea has been used in the learning to rank approach introduced in [33], where it is shown that different types of privileged information, such as bounding boxes, attributes, text, and annotator rationales [7] can be used for learning a better classifier for object recognition.”
Leveraging Visual Question Answering for Image-Caption Ranking (2016, background): “ple images (zero-shot learning [30,39]) and improving human-machine communication where a user can explain the target concept during image search [29,26], or give a classier an explanation of labels [10,40]. Recent works also explore using word embeddings [47] and freeform text [12] as representatio”
Active Multitask Learning Using Both Latent and Supervised Shared Topics (2014, background): “One could additionally actively query for rationales [42, 12] and further improve the predictive performance.”
Fine-Grained Crowdsourcing for Fine-Grained Recognition (2013, background): “by asking humans to directly provide annotation rationales [9], to label features in NLP tasks [10], to describe the differences between pairs of images [20], or to perform tasks that are parts of the machine pipeline [23].”
Similarity Comparisons for Interactive Fine-Grained Categorization (2014, background): “Some recent works use similarity in feature space [18, 3]; others rely on human judgment to quantify similarity for classifying attributes or clustering categories [27, 7, 15, 21].”
TED: Teaching AI to Explain its Decisions (2018, background): “The second result has been observed by other researchers who have suggested adding “rationales” to improve classifier performance, but not for explainability [8, 11, 25, 29, 32, 37–39].”
Cross-Domain Image Retrieval with a Dual Attribute-Aware Ranking Network (2015, background): “The work by Donahue and Grauman [7] demonstrates that richer supervision conveying annotator rationales based on visual attributes, can be considered as a form of privileged information [42].”
Understanding Objects in Detail with Fine-Grained Attributes (2014, background): “Attributes can also be used to transfer expert knowledge to a computer vision system in zero-shot learning [25] [43], in finding relations between object categories [34] [42], in incorporating annotator's rationales and other forms of feedback [8] [35].”
Discovering Attribute Shades of Meaning with the Crowd (2015, background): “Furthermore, recent work using crowds to isolate informative spatial regions (Donahue and Grauman, 2011; Deng et al, 2013) has a different purpose (finegrained image classification) and takes an entirely different approach (explicitly asking labelers to outline the regions needed to make their label”
Learning Credible Deep Neural Networks with Rationale Regularization (2019, methodology): “model’s attention, enabling it to tease apart useful evidence from noises and pushing it to pay more attention to relevant features. Rationales have been applied to the training process of SVMs [15], [17] to enhance predictive performance. Another beneﬁt of rationales is that they require little eff”
The Ignorant Led by the Blind: A Hybrid Human–Machine Vision System for Fine-Grained Categorization (2014, methodology): “A number of exciting active learning algorithms that incorporate new types of human interactivity have come about in recent years (Vijayanarasimhan and Grauman 2009, 2011; Donahue and Grauman 2011; Vondrick and Ramanan 2011; Settles 2008; Parkash and Parikh 2012; Branson et al. 2011).”
What has my classifier learned? Visualizing the classification rules of bag-of-feature model by support region detection (2012, background): “However, its purpose is to incorporate the annotator’s rationale to design a better classifier, while our aim is to visualize the “rationales of a classifier” learned purely from a set of training samples.”
WhittleSearch: Image search with relative attribute feedback (2012, background): “Comparative information can also be used to train object recognition systems, by stating similarity between object categories [26], comparing their attributes [20], or explaining attributes that make them different [4].”
Learning to Rank Using Privileged Information (2013, methodology): “In [7] it was shown that these can act as additional sources of information during training, as long as the rationales can be expressed in the same data representation as the original data (e.”