Efficient Exploration in Reinforcement Learning

By J. Langford. 2017

217 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Control and Systems Engineering

Citations in the corpus

Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models (2018, background): “Isolating epistemic uncertainty is especially useful for directing exploration [Thrun, 1992], although we leave this for future work.”
Parameter Space Noise for Exploration (2017, methodology): “We refer the reader to [32, 16] for details on the respective technique.”
Supervision via competition: Robot adversaries for learning tasks (2016, background): “active learning has ranged from curriculum learning [19], selectively filtered uncommon or interesting data [20], to directed exploration in reinforcement learning (see [21]).”
DORA The Explorer: Directed Outreaching Reinforcement Action-Selection (2018, background): “The most commonly used exploration bonus is based on counting (Thrun, 1992) – for each pair (s, a), store a counter C (s, a) that indicates how many times the agent performed action a at state s so far.”
An Introduction to Deep Reinforcement Learning (2018, unknown): “The exploration-exploitation dilemma is a well-studied tradeoff in RL (e.g., Thrun, 1992).”
Simplifying Deep Reinforcement Learning via Self-Supervision (2021, methodology): “Efﬁcient exploration is one of the fundamental challenges in reinforcement learning (Thrun 1992).”
Improving PILCO with Bayesian Neural Network Dynamics Models (2016, methodology): “Since we use probabilistic models throughout planning we can use uncertainty-directed exploration which is much more efficient and informed than undirected random exploration techniques such as epsilon-greedy or Boltzmann distributions [15].”
Deep learning in medical image registration: a review (2019, background): “Reinforcement learning (RL) is a type of machine learning that focused on predicting the best actions to take given its current state in an environment (Thrun 1992).”
Offline Reinforcement Learning with Pseudometric Learning (2021, background): “Note that the concept of a bonus is overloaded in RL; it typically applies to exploration strategies (Schmidhuber, 1991; Thrun, 1992; Bellemare et al., 2016).”
VIME: Variational Information Maximizing Exploration (2016, background): “Other metrics for curiosity have also been proposed, including prediction error [10, 35], prediction error improvement [36], leverage [14], neuro-correlates [37], and predictive information [38].”
Understanding the origin of information-seeking exploration in probabilistic objectives for control (2021, unknown): “…from machine learning and reinforcement learning (Kaelbling et al., 1998; Oudeyer & Kaplan, 2009; Schmidhuber, 1991, 1999, 2007; Sun et al., 2011; Thrun, 1992; Thrun & Möller, 1991), cognitive science and neuroscience (Cohen et al., 2007; Daw, O’doherty, Dayan, Seymour, & Dolan, 2006; Dayan &…”
Reinforcement learning for control: Performance, stability, and deep approximators (2018, background): “Exploration methods can be grouped into undirected and directed exploration (Thrun, 1992).”
Teacher–Student Curriculum Learning (2017, background): “Keylock MDP [5] is a simple task that is challenging for current reinforcement learning methods.”
Model-Based Active Exploration (2018, background): “In the simple cases, undirected exploration techniques (Thrun, 1992) such as adding random noise to the actions of the policy might be sufficient to break out of metastability.”
CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning (2018, background): “First, given that exploration is crucial for RL [41] and even more so in MARL with larger state and joint action spaces, how should agents explore to learn both individual goal attainment and cooperation for others’ success? Uniform random exploration is common in deep MARL [8] but can be highly ine”