The benefits and challenges of collecting richer object annotations

By Ian Endres, Ali Farhadi, Derek Hoiem, D.A. Forsyth. 2010

46 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

Extreme Clicking for Efficient Object Annotation (2017, background): “A qualification test is a good mechanism for enhancing the quality of crowd-sourcing data and for filtering out bad annotators and spammers [1, 19, 33, 37].”
Annotator rationales for visual recognition (2011, background): “This includes gathering fully segmented [19], poseannotated [1], or “attribute”-labeled images [8, 12, 13, 17].”
Crowdsourcing in Computer Vision (2016, unknown): “The approaches of Vondrick and Ramanan [2011], Vijayanarasimhan and Grauman [2012], Fathi et al. [2011] and others additionally incorporate active learning, where the annotation interfaces learns to query frames that, if annotated, would produce the largest expected change in the estimated object tr”
Y Qualification Main task Y Return N N Y User qualified ? N Feedback Pass ? Pass ? Annotator training Annotating images (2017, background): “A qualification test is a good mechanism for enhancing the quality of crowd-sourcing data and for filtering out bad annotators and spammers [1, 19, 33, 37].”
Discovering Attribute Shades of Meaning with the Crowd (2015, background): “Since annotators may disagree about the attribute label for an image (Farhadi et al, 2009; Endres et al, 2010; Patterson and Hays, 2012; Curran et al, 2012), the norm is to take the majority vote label (and discard the image if votes are too split).”
Understanding Objects in Detail with Fine-Grained Attributes (2014, background): “This section discusses the nuances involved in collecting a large set of detailed object annotations [10] and a methodology to do so efficiently.”
Multiclass recognition and part localization with humans in the loop (2011, background): “Beyond traditionally studied basic-level categories, these interests have led to progress in transfer learning and learning from fewer training examples [7, 8, 10, 15, 21], larger scale computer vision algorithms that share processing between tasks [15, 16], and new methodologies for data collection”