AGIL: Learning Attention from Human for Visuomotor Tasks

By Ruohan Zhang, Zhuode Liu, Luxin Zhang, Jake Alden Whritner, Karl S. Muller, Mary M. Hayhoe, Dana H. Ballard. 2018

85 citations recorded by the source corpus.

When intelligent agents learn visuomotor behaviors from human demonstrations, they may benefit from knowing where the human is allocating visual attention, which can be inferred from their gaze. A wealth of information regarding intelligent decision making is conveyed by human gaze allocation; hence, exploiting such information has the potential to improve the agents’ performance. With this motivation, we propose the AGIL (Attention Guided Imitation Learning) framework. We collect high-quality human action and gaze data while playing Atari games in a carefully controlled experimental setting. Using these data, we first train a deep neural network that can predict human gaze positions and visual attention with high accuracy (the gaze network) and then train another network to predict human actions (the policy network). Incorporating the learned attention model from the gaze network into the policy network significantly improves the action prediction accuracy and task performance.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation (2020, methodology): “The settings of the Gaussian blurring ﬁlters are listed below: • 1 Augmentation: ◦ ﬁlter size: 5, σ : 5 AGIL [50] was designed to utilize saliency map collected via human gaze.”
Leveraging Human Guidance for Deep Reinforcement Learning Tasks (2019, methodology): “Existing works have proposed to use a learned gaze prediction model to guide the process of learning a human policy [Li et al. , 2018; Zhang et al. , 2018; Kim et al. , 2018].”
Using Human Gaze to Improve Robustness Against Irrelevant Objects in Robot Manipulation Tasks (2020, background): “Thereisa research to use gaze prediction for imitation learning in Atari games [39].”
HARMONIC: A multimodal dataset of assistive human–robot collaboration (2018, background): “…has been work in learning and leveraging human policies (using keyboard input) (Reddy et al. 2018b,a) and attention models (using keyboard input and eye gaze) (Zhang et al. 2018) for both assisted and shared robot control in Atari games in an arcade learning environment (Bellemare et al. 2012).”
A Survey on Vision--Language--Action Models for Embodied AI (2024, unknown): “We hope this survey captures the rapidly evolving landscape of embodied AI and inspires future research.”
Efficiently Guiding Imitation Learning Algorithms with Human Gaze (2020, methodology): “Zhang et al. (2018) train a gaze-prediction network using a stack of 4 image frames denoting the game state, optical flow and saliency (Itti, Koch, and Niebur 1998) as input.”
Human Gaze Assisted Artificial Intelligence: A Review (2020, methodology): “Convolutional neural networks remain the most popular tool for predicting human attention [Li et al. , 2018; Zhang et al. , 2018a; Palazzi et al. , 2018; Deng et al. , 2019].”
Reinforcement Learning with Attention that Works: A Self-Supervised Approach (2019, unknown): “Although previous attempts to integrate attention with RL have been made, these attempts have largely used hand-crafted features as inputs to the attention model [28,29].”
Free-Lunch Saliency via Attention in Atari Agents (2019, methodology): “The work [46] also uses human attention data but brings a different approach: they have collected a dataset of human gaze positions with an eye tracker, trained an encoderdecoder architecture to predict human gaze positions using that data, and finally used it to train an agent with imitation learni”
Transformers for One-Shot Visual Imitation (2020, background): “Indeed, humans use attention mechanisms to create context driven representations [13], and directly supervising policies with human attention can dramatically increase task performance [14].”