Guiding a Reinforcement Learner with Natural Language Advice: Initial Results in RoboCup Soccer

By Gregory Kuhlmann and Peter Stone and Raymond J. Mooney and Shavlik. 2004

149 citations recorded by the source corpus.

Subfields: Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Dialog-based Language Learning (2016, background): “ck. There is work on the the use of feedback and dialog for learning, notably for collecting knowledge to answer questions [8, 17], the use of natural language instruction for learning symbolic rules [10, 5] and the use of binary feedback (rewards) for learning parsers [3]. Another setting which use”
Inner Monologue: Embodied Reasoning through Planning with Language Models (2022, background): “Some works have been trained directly for language-based interaction in robotic tasks [67, 68, 69, 70].”
Learning for Semantic Parsing with Statistical Machine Translation (2006, methodology): “, 1995) and advice taking (Kuhlmann et al., 2004).”
Teaching Machines to Describe Images via Natural Language Feedback (2017, background): “While this is possible in limited domains such as in navigating a maze [22] or learning to play a soccer game [15], it can hardly scale to the real scenarios with large action spaces requiring versatile language feedback.”
Integrating reinforcement learning with human demonstrations of varying ability (2011, background): “, the learning agent must have natural language processing ab lities [10]).”
A comprehensive survey on safe reinforcement learning (2015, background): “Within the RL framework, Sato et al. (2002) propose an approach that directly optimizes an objective function defined as a linear combination of the mean and the variance of the return.”
TAMER: Training an Agent Manually via Evaluative Reinforcement (2008, background): “[11] created a domain-specific natural language interface for giving advice to a reinforcement learner.”
How Humans Teach Agents - A New Experimental Perspective (2012, background): “Other modes, though given less attention, have also been studied: learning from advice [21, 24], learning from numeric feedback or reward [15, 18, 37, 40], and allowing the human to shape the agent’s learning environment, facilitating the learning process [32, 38].”
Bridging the Gap: Converting Human Advice into Imagined Examples (2019, background): “Kuhlmann et al. (2004) took advantage of the restricted nature of the target task (RoboCup soccer) to successfully translate natural language advice into a formal semantic representation directly usable by the agent.”
Reinforcement Learning Via Practice and Critique Advice (2010, unknown): “For example , rules in the form of programming-language constructs (Maclin and Shavlik 1996), logical rules derived from a constrained natural language (Kuhlmann et al. 2004), and rules about action utilities and preferences (Maclin et al. 2005b; 2005a).”
Interactively shaping agents via human reinforcement: the TAMER framework (2009, background): “[11] created a domain-specific natural language interface for giving advice to a reinforcement learner.”
PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards (2020, background): “Kuhlmann et al. (2004) generate rules in a custom language from a set of natural language instructions.”
Learning a Compositional Semantic Parser using an Existing Syntactic Parser (2009, unknown): “Competition and manually translated into English (Kuhlmann et al., 2004).”
Learning From Explanations Using Sentiment and Advice in RL (2017, background): “Several connect conditions to higher-level actions that are defined by the researcher instead of primitive actions [8], [10], [12].”
Learning to Learn Semantic Parsers from Natural Language Supervision (2019, background): “, 2015) and guiding policy learning using high-level human advice (Kuhlmann et al., 2004; Squire et al., 2015; Harrison et al., 2017).”