Prioritized sweeping: Reinforcement learning with less data and less time

By Andrew W. Moore, Christopher G. Atkeson. 2004

621 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Learning Parameterized Skills (2012, background): “The second domain in which we evaluate TD(0)-∆π is a rod positioning task [49], which consists of a discretized space containing a rod, obstacles, and a target.”
Distributed Prioritized Experience Replay (2018, background): “Prioritized experience replay (Schaul et al., 2016) extends classic prioritized sweeping ideas (Moore & Atkeson, 1993) to work with deep neural network function approximators.”
Prioritized Experience Replay (2015, background): “Prioritized sweeping (Moore & Atkeson, 1993; Andre et al., 1998) selects which state to update next, prioritized according to the change in value, if that update was executed.”
Deep learning in neural networks: An overview (2014, neutral): “…Kaelbling, Littman, & Cassandra, 1995; Lagoudakis & Parr, 2003; Maei & Sutton, 2010; Mahadevan, 1996; Meuleau, Peshkin, Kim, & Kaelbling, 1999; Moore & Atkeson, 1993; Morimoto & Doya, 2000; Peng & Williams, 1996; Prokhorov &Wunsch, 1997; Rummery &Niranjan, 1994; Santamaría, Sutton, & Ram,…”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, positive): “Although replay selection strategies have not shown as much benefit for the supervised learning scenario, they have significantly improved the performance and efficiency of training in reinforcement learning agents (Moore & Atkeson, 1993).”
Task-Free Continual Learning (2018, methodology): “A common DRL technique, known as “prioritized sweeping”, is to sample experiences with a large error more often than others [23].”
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification (2020, background): “Incremental task acquisition enjoys a long tradition in the RL community [100], and more recently the CLEAR approach mixes on-policy learning for plasticity with off-policy learning from replayed experiences to encourage stability with respect to tasks acquired in the past [73].”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “The parameters or weights of C are trained to maximize reward by a standard RL method such as Q-learning or similar methods [17, 292, 293, 172, 254, 212, 262, 10, 122, 188, 157, 281, 26, 216, 197, 272, 311, 9, 163, 174, 22, 27, 2, 137, 276, 156, 284].”
Dyna-Style Planning with Linear Function Approximation and Prioritized Sweeping (2008, methodology): “The original papers on the Dyna architecture and most subsequent extensions (e.g., Singh 1992; Peng & Williams 1993; Moore & Atkeson 1993; Kuvayev & Sutton 1996) assumed a Markov environment with a tabular representation of states.”
Combating the Compounding-Error Problem with a Multi-step Model (2019, methodology): “Note that prior works on ensemble methods in model-based RL exist Moore and Atkeson [1993]; Kurutach et al. [2018], but only consider combinations of one-step models.”
Locality Sensitive Sparse Encoding for Learning World Models Online (2024, methodology): “Search control defines how the agent queries the model, and some commonly used ones include predecessor (Moore & Atkeson, 1993), on-policy (Janner et al., 2019) and hill-climbing (Pan et al., 2019).”
Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial (2020, background): “Prioritized sweeping [73] assigns priorities to state-action pairs according to the changes in their estimated values.”
Neural Networks and Deep Learning (2018, positive): “It has been suggested in [335] that the approach presented in the paper can be improved with other well-known ideas such as prioritized sweeping [343].”
Deep Reinforcement Learning for List-wise Recommendations (2017, background): “More-over, we leverage prioritized sampling strategy [15] to assist the framework learning from the most important historical transitions.”
A Comprehensive Survey of Multiagent Reinforcement Learning (2008, methodology): “, model-based methods based on dynamic programming [29]– [31], model-free methods based on online estimation of valu e functions [32]–[35], and model-learning methods that esti ma e a model, and then learn using model-based techniques [36], [37].”
A Decision-Theoretic Model of Assistance (2007, methodology): “However, using incremental dynamic programming methods such as prioritized sweeping (Moore & Atkeson, 1993) can alleviate much of the computational cost.”
Exploration Bonuses and Dual Control (1996, methodology): “Moore & Atkeson (1993) achieve something similar to exploration bonuses with an agent that uses a different model of the environment from the one actually experienced.”