Learning behaviors via human-delivered discrete feedback: modeling implicit feedback strategies to speed up learning

By Robert Tyler Loftin, Bei Peng, James MacGlashan, Michael L. Littman, Matthew E. Taylor, Jeff Huang, David L. Roberts. 2015

109 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Science Applications, Reinforcement Learning, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Peter Stone
3. Li Fei-Fei
4. Silvio Savarese
5. Pieter Abbeel
6. Jianfeng Gao
7. Trevor Darrell
8. Yuke Zhu

In Computer Science Applications

1. Peter Stone
2. Sanmit Narvekar
3. Hoang Minh Le
4. Hal Daumé Iii
5. Yisong Yue
6. Bei Peng
7. Li Fei-Fei
8. Nan Jiang

In Reinforcement Learning

1. Sergey Levine
2. Peter Stone
3. Nan Jiang
4. Alekh Agarwal
5. Matthew E. Taylor
6. Yuke Zhu
7. Ajay Mandlekar
8. Faraz Torabi

In Human-Computer Interaction

1. Ruohan Zhang
2. Lin Guan
3. Ajay Mandlekar
4. Dana H. Ballard
5. Peter Stone
6. Silvio Savarese
7. Hal Daumé Iii
8. Siddhartha S. Srinivasa

Citations in the corpus

• Human-in-the-Loop Imitation Learning using Remote Teleoperation (2020, background): “One paradigm is Reinforcement Learning (RL) with human feedback [40], where a human provides rewards during agent training [8, 11, 24, 25, 34, 36], but this suffers from the same limitations as IRL due to the need for extensive agent interaction.”
• Leveraging Human Guidance for Deep Reinforcement Learning Tasks (2019, methodology): “The nature of the feedback could potentially vary across tasks and trainers [ Loftin et al. , 2016 ] , and can be altered by instruction given to the trainers [ Cederborg et al. , 2015 ] .”
• Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces (2017, background): “Most closely related to the work presented here, several methods have recently been proposed that allow an agent to learn from interaction with a non-expert human (Thomaz and Breazeal 2006; MacGlashan et al. 2014; Loftin et al. 2014; 2016; Peng et al. 2016).”
• Hierarchical Imitation and Reinforcement Learning (2018, background): “The primary other way to elicit weaker demonstration feedback is with preference-based or gradient-based learning, studied by Fürnkranz et al. (2012), Loftin et al. (2016), and Christiano et al. (2017).”
• Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey (2020, methodology): “It uses the goal-directed Strategy-Aware Bayesian Learning (SABL) algorithm (Loftin et al., 2016) for learning from human feedback.”
• Interactive reinforcement learning for task-oriented dialogue management (2016, methodology): “In this work, we assume that feedback is regular and consistent and we have not explored more sophisticated models of human-provided explicit and implicit feedback (Loftin et al. (2016)).”
• Robots That Use Language (2020, background): “A bottle consists of a bottle and a cap, and a person referencing “the bottle” may mean both, or they may say, “Grab the bottle and then turn the cap,”referring”
• Learning from Interventions: Human-robot interaction as both explicit and implicit feedback (2020, background): “multi-task learning framework, and thus learn from multiple experts with perhaps different thresholds and deduce which objectives they are biased towards [27].”