Task and context determine where you look.

By Constantin A. Rothkopf, Dana H. Ballard, Mary M. Hayhoe. 2016

424 citations recorded by the source corpus.

The deployment of human gaze has been almost exclusively studied independent of any specific ongoing task and limited to two-dimensional picture viewing. This contrasts with its use in everyday life, which mostly consists of purposeful tasks where gaze is crucially involved. To better understand deployment of gaze under such circumstances, we devised a series of experiments, in which subjects navigated along a walkway in a virtual environment and executed combinations of approach and avoidance tasks. The position of the body and the gaze were monitored during the execution of the task combinations and dependence of gaze on the ongoing tasks as well as the visual features of the scene was analyzed. Gaze distributions were compared to a random gaze allocation strategy as well as a specific "saliency model." Gaze distributions showed high similarity across subjects. Moreover, the precise fixation locations on the objects depended on the ongoing task to the point that the specific tasks could be predicted from the subject's fixation data. By contrast, gaze allocation according to a random or a saliency model did not predict the executed fixations or the observed dependence of fixation locations on the specific task.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control (2021, background): “Such coordination coupled with the ability to switch visual attention at different stages of a task allows humans to adaptively direct their movements at taskrelevant objects and be invariant to the objects’ absolute spatial locations [3, 12, 29].”
Eye guidance in natural vision: Reinterpreting salience (2011, neutral): “, 1999), whereas before beginning a task (such as sandwich making) the distribution of fixations between task-relevant and -irrelevant objects is about equal (Hayhoe et al., 2003; Rothkopf et al., 2007).”
Gaze allocation in natural stimuli: Comparing free exploration to head-fixed viewing conditions (2009, background): “In contrast to truly real-world experiments, the same trial can be replicated and the environment can be controlled, allowing quantitative manipulations and thus assessment of the interaction between task, stimulus, and context (Rothkopf et al., 2007).”
Visual Attention Guided Deep Imitation Learning (2017, background): “Consider [8] in which they show that humans look at the edges of obstacles to avoid them, and the centers of targets to approach them.”
AGIL: Learning Attention from Human for Visuomotor Tasks (2018, background): “[35] have shown that varying task instructions drastically alters the gaze distributions on different categories of objects (e.”
Get real! Resolving the debate about equivalent social stimuli (2009, background): “In fact, it appears that task demands can completely override the effects of visual saliency in complex visual displays (Einhauser, Rutishauser, & Koch, 2008 ; Foulsham & Underwood, 2008 ; Nystrom & Holmqvist, 2008 ; Rothkopf, Ballard, & Hayhoe, 2007 ).”
Attention, learning and the value of information (2012, background): “Such a mechanism can be prohibitively slow in complex tasks, and may erroneously assign credit to irrelevant steps (Rothkopf et al., 2007).”
Transformers for One-Shot Visual Imitation (2020, background): “Indeed, humans use attention mechanisms to create context driven representations [13], and directly supervising policies with human attention can dramatically increase task performance [14].”
Efficiently Guiding Imitation Learning Algorithms with Human Gaze (2020, background): “Human attention in the form of eye gaze has been known to encode top-down attention versus bottom-up salience when performing goal directed tasks (Land 2009; Hayhoe and Ballard 2005; Rothkopf, Ballard, and Hayhoe 2007; Tatler et al. 2011).”