Scaling Reinforcement Learning toward RoboCup Soccer

By Peter Stone, Richard S. Sutton. 2001

218 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Benchmarks for Deep Off-Policy Evaluation (2021, background): “Outside of OPE, standardized benchmark suites have led to considerable standardization and progress in RL (Stone & Sutton, 2001; Dutech et al., 2005; Riedmiller et al., 2007).”
Guiding a Reinforcement Learner with Natural Language Advice: Initial Results in RoboCup Soccer (2004, methodology): “Giving Advice to the Keeper Previously, Stone and Sutton (2001) achieved successful learning results in the keepaway domain using a reinforcement learning algorithm called episodic SMDP Sarsa( ) (Sutton & Barto 1998).”
Evolving Soccer Keepaway Players Through Task Decomposition (2005, methodology): “A variant based on the the RoboCup soccer simulator was introduced for the purposes of studying multi-agent reinforcement learning (Stone and Sutton, 2001).”
Deep Multi-Agent Reinforcement Learning for Decentralized Continuous Cooperative Control (2020, background): “Presumably due to its focus on real-world robotic control, RoboCup Soccer Simulation (Kitano et al., 1997; Stone & Sutton, 2001; Riedmiller et al., 2009) does not currently feature an easily conﬁgurable software interface for MARL, nor suitable AI-controlled benchmark opponents.”
Nash Q-Learning for General-Sum Stochastic Games (2003, methodology): “directly to the multiagent context, its ease of application does, and the method has been employed in such multiagent domains as robotic soccer (Stone and Sutton, 2001, Balch, 1997), predator-andprey pursuit games (Tan, 1993, De Jong, 1997, Ono and Fukumoto, 1996), and Internet pricebots (Kephart…”
A Neuroevolution Approach to General Atari Game Playing (2014, methodology): “Of the algorithms used in this paper, HyperNEAT has been previously applied to the games of Robocup Keepaway Soccer [45], [51], Checkers [15], multiagent predator prey [10], and quadruped locomotion [7].”
Improving Action Selection in MDP's via Knowledge Transfer (2005, background): “While largestate sets have seen much work in recent research (Tesauro 1994; Crites & Barto 1996; Stone & Sutton 2001), largeactionsets have been explored to but a limited extent (Santamaria, Sutton, & Ram 1997; Gaskett, Wettergreen, & Zelinsky 1999).”
Recent Advances in Hierarchical Reinforcement Learning (2003, methodology): “See Sutton et al. [73] and Precup [52] for details on these and other option-related algorithms and illustrations of their performance.”
Reinforcement learning for sensing strategies (2004, background): “Furthermore, the system is highly non-linear and it is not clear how the different uncertainties affect the outcome of the actions, for example, of grabbing the ball or kicking it toward the goal.”
Evolving Keepaway Soccer Players through Task Decomposition (2003, methodology): “For example, Stone and Sutton implemented keepaway in the RoboCup soccer simulator [14].”
Giving Advice about Preferred Actions to Reinforcement Learners Via Knowledge-Based Kernel Regression (2005, methodology): “We call this new task BreakAway; it is similar in spirit to the KeepAway challenge problem of Stone and Sutton (2001).”
Scalable Planning and Learning for Multiagent POMDPs (2014, background): “…this is a negative result, and there is no guarantee of convergence for FT-FV-POMCP, in practice this need not be a problem; many reinforcement learning techniques that can diverge (e.g., neural networks) can produce high-quality results in practice, e.g., (Tesauro, 1995; Stone and Sutton, 2001).”
Nash Q-Learning for General-Sum Stochastic Games (2003, methodology): “directly to the multiagent context, its ease of application does, and the method has been employed in such multiagent domains as robotic soccer (Stone and Sutton, 2001, Balch, 1997), predator-andprey pursuit games (Tan, 1993, De Jong, 1997, Ono and Fukumoto, 1996), and Internet pricebots (Kephart…”
Adaptive Neural Network Control of AUVs With Control Input Nonlinearities Using Reinforcement Learning (2017, background): “In [54], the “keepers” of one soccer team were trained to learn when to hold or pass the ball.”
HyperNEAT-GGP: a hyperNEAT-based atari general game player (2012, background): “ley [18] who focused on the RoboCup Keepaway Soccer domain [14].”
Reinforcement Learning for RoboCup Soccer Keepaway (2005, methodology): “5.3.1 Limited Vision Limited vision introduces two challenges with respect to the complete vision setup of Section 5.1 and Stone and Sutton (2001).”