Human Gaze Assisted Artificial Intelligence: A Review

By Ruohan Zhang, Akanksha Saran, Yifeng Zhu, Sihang Guo, Scott Niekum, Dana H. Ballard, Mary M. Hayhoe. 2020

82 citations recorded by the source corpus.

Human gaze reveals a wealth of information about internal cognitive state. Thus, gaze-related research has significantly increased in computer vision, natural language processing, decision learning, and robotics in recent years. We provide a high-level overview of the research efforts in these fields, including collecting human gaze data sets, modeling gaze behaviors, and utilizing gaze information in various applications, with the goal of enhancing communication between these research areas. We discuss future challenges and potential applications that work towards a common goal of human-centered artificial intelligence.

Subfields: Computer Vision and Pattern Recognition, Urban Studies, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Lin Guan
2. Subbarao Kambhampati
3. Sihang Guo
4. Ruohan Zhang

In Human-Computer Interaction

1. Ruohan Zhang
2. Sihang Guo
3. Lin Guan

Citations in the corpus

• Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation (2020, background): “Leveraging human guidance for RL tasks has been extensively studied in the context of imitation learning (learning from demonstration) [37], inverse reinforcement learning [29, 1], reward shaping [28], learning from human preference [7, 22], and learning from saliency information provided by human [”
• A Survey on Vision--Language--Action Models for Embodied AI (2024, unknown): “We hope this survey captures the rapidly evolving landscape of embodied AI and inspires future research.”