Training Object Class Detectors from Eye Tracking Data

By Dim P. Papadopoulos, Alasdair D. F. Clarke, Frank Keller, Vittorio Ferrari. 2014

153 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Extreme Clicking for Efficient Object Annotation (2017, background): “Interestingly, 1s visual search is consistent with earlier findings [18, 46].”
What's the Point: Semantic Segmentation with Point Supervision (2015, background): “Many forms of supervision have been explored, such as eye tracks [25], free-form squiggles [41], noisy web tags [1], or heterogeneous annotations (a large number of image-level labels and a small number of full segmentation masks) [17].”
Latent Embeddings for Zero-Shot Classification (2016, background): “Combining visual information with attributes [7, 8, 11, 17, 20, 28, 27] has also supported fine grained classification.”
A Comprehensive Survey of Deep Learning for Image Captioning (2018, background): “Thus, gaze mechanisms have already shown their potential performances in eye-based user modeling [18, 35, 109, 122, 124], object localization [100] or recognition [67] and holistic scene understanding [158, 160].”
Weakly- and Semi-Supervised Panoptic Segmentation (2018, methodology): “Bounding boxes take only 7 seconds to draw using the labelling method of [7], and image-level tags an average of 1 second per class [8].”
Seeing with Humans: Gaze-Assisted Neural Image Captioning (2016, methodology): “Consequently, gaze has successfully been used for tasks such as eye-based user modeling [1], [2], [3], [4], [5] and opened up new opportunities to further advance human-machine collaboration – collaborative human-machine vision systems in which part of the processing is carried out by the machine…”
Rendering of Eyes for Eye-Shape Registration and Gaze Estimation (2015, background): “Therefore they are important for a range of applications including gaze-based human-computer interaction [2], visual behavior monitoring [3], and – more recently – collaborative human-computer vision systems [4, 5].”
Y Qualification Main task Y Return N N Y User qualified ? N Feedback Pass ? Pass ? Annotator training Annotating images (2017, background): “Interestingly, 1s visual search is consistent with earlier findings [18, 46].”
Human Gaze Assisted Artificial Intelligence: A Review (2020, methodology): “In recent years, data-driven approaches became more popular as large-scale eye-tracking datasets became available for images [Papadopoulos et al. , 2014; Li et al. , 2014; Xu et al. , 2014; Bylinskii et al. , 2015b; Bylinskii et al. , 2015a; Krafka et al. , 2016], videos [Mathe and Sminchisescu,…”
We Don’t Need No Bounding-Boxes: Training Object Class Detectors Using Only Human Verification (2016, methodology): “Other types of data, such as text from web pages or newspapers [3, 15, 22, 33] or eye-tracking data [35], have also been used as a weak annotation signal to train object detectors.”
Eye tracking assisted extraction of attentionally important objects from videos (2015, background): “In a free viewing task, eye movements are biased towards high level semantics [22, 29] in static and dynamic scenes.”
SALICON: Saliency in Context (2015, background): “Even if POET, the largest dataset we know by far, contains 6,270 images and is only viewed by 5 subjects [21].”
What are the Visual Features Underlying Human Versus Machine Vision? (2017, background): “Researchers have found that in cases like these human perceptual judgments can augment the performance of vision models [35], resulting in significant gains in face recognition and localization [24, 2, 14], action recognition [31], object detection and segmentation [20, 26, 32].”
TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking (2015, background): “The data sets differ in several key parameters [5], including: the number and style of images/video chosen, the number of subjects, the number of views per image, the subjects distance from the screen, eye trackers, the exact task the subjects were given (free viewing [16], object search [26], perso”
Attentive Systems: A Survey (2017, background): “Papadopoulos et al. (2014) train object class detectors from eye tracking data in order to pursue the paradigm ‘learning object detectors while watching TV.”
Discovering Class-Specific Pixels for Weakly-Supervised Semantic Segmentation (2017, background): “Among these supervisions image-level labels are easiest to collect – almost 1 second per class or object-category [22] – and also are amenable to webly-supervised learning where one can download millions of images of new object categories from the Internet for training.”
Large-Scale Interactive Object Segmentation With Human Annotators (2019, methodology): “A flurry of techniques have been explored to annotate the location of objects in images: bounding box annotations [8], clicks on the object extremes [26, 22], clicks on the object centre [3, 27], bounding box followed by edits to a generated segment [4, 30], scribbles [3, 18], hierarchical super-pix”