Studying Relationships between Human Gaze, Description, and Computer Vision

By Kiwon Yun, Yifan Peng, Dimitris Samaras, Gregory J. Zelinsky, Tamara L. Berg. 2013

88 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors (2022, background): “A clear example of this gap can be observed in person and face generation, where a dissonance is present between the importance of face pixels from the human perspective and the loss applied over the whole image [28, 66].”
Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels (2015, background): “Human-centric annotations [3] exhibit noise that is highly structured and shows statistical dependencies on the data [3, 14, 55].”
What Do Different Evaluation Metrics Tell Us About Saliency Models? (2016, methodology): “Consider a detection application of saliency such as object and motion detection, surveillance, localization and mapping, and segmentation [1], [17], [27], [28], [43], [50], [59], [91].”
Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention (2020, background): “While neural networks are able to learn meaningful attention mechanisms using only supervision received for the target task, the addition of human gaze information has been shown to be beneﬁcial in many cases [32, 58, 80, 84].”
Action from Still Image Dataset and Inverse Optimal Control to Learn Task Specific Visual Scanpaths (2013, background): “Human gaze pattern annotations have been collected for both static images[11, 13, 14, 12, 26, 18] and for video[19, 23, 15], see [24] for a recent overview.”
Eye tracking assisted extraction of attentionally important objects from videos (2015, background): “Automated extraction of objects in a video sequence can benefit several applications related to annotation, compression, summarization, search and retrieval.”
Attentive Systems: A Survey (2017, background): “There aremanyworks that leverage human gaze to facilitate tasks of interest (Lee et al. 2012; Mishra et al. 2012; Xu et al. 2015; Yun et al. 2013).”
Salient Object Detection: A Benchmark (2015, background): “, [134], [137]–[139]), and even object and scene classification (e.”
Seeing with Humans: Gaze-Assisted Neural Image Captioning (2016, methodology): “Gaze-related features were used for object recognition [13], or incorporated into the image-based object recognition pipeline [3], [30], [31].”
Training Object Class Detectors from Eye Tracking Data (2014, methodology): “[52] collect eye movement data for a 1,000 image subset of Pascal VOC 2008; three observers performed a three second free-viewing task.”