Self-improving reactive agents based on reinforcement learning, planning and teaching

By Longxin Lin. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Sample Efficient Actor-Critic with Experience Replay (2016, methodology): “Experience replay (Lin, 1992) has gained popularity in deep Q-learning (Mnih et al., 2015; Schaul et al., 2016; Wang et al., 2016; Narasimhan et al., 2015), where it is often motivated as a technique for reducing sample correlation.”
Learning Continuous Control Policies by Stochastic Value Gradients (2015, methodology): “To increase data-efficiency, we construct an off-policy, experience replay [15, 29] algorithm that uses models and value functions, SVG(1) with Experience Replay (SVG(1)-ER).”
Addressing Function Approximation Error in Actor-Critic Methods (2018, background): “This update can be applied in an off-policy fashion, sampling random mini-batches of transitions from an experience replay buffer (Lin, 1992).”
When to use parametric models in reinforcement learning? (2019, methodology): “Experience replay [Lin, 1992] refers to storing previously observed transitions to replay later for additional updates to the predictions and policy.”
Distributed Prioritized Experience Replay (2018, methodology): “Prioritized Experience Replay Experience replay (Lin, 1992) has long been used in reinforcement learning to improve data efficiency.”
Rainbow: Combining Improvements in Deep Reinforcement Learning (2017, background): “At each step, based on the current state, the agent selects an action (cid:15) -greedily with respect to the action values, and adds a transition ( S t , A t , R t +1 , γ t +1 , S t +1 ) to a replay memory buffer (Lin 1992), that holds the last million transitions.”
Prioritized Experience Replay (2015, background): “Experience replay (Lin, 1992) addresses both of these issues: with experience stored in a replay memory, it becomes possible to break the temporal correlations by mixing more and less recent experience for the updates, and rare experience will be used for more than just a single update.”
Deep Reinforcement Learning with Double Q-Learning (2015, methodology): “The target used by DQN is then For the experience replay (Lin, 1992), observed transitions are stored for some time and sampled uniformly from this memory bank to update the network.”
Neural Approaches to Conversational AI (2018, neutral): “le popular, in practice, Q-learning can be unstable and requires many samples before reaching a good approximation of Q. Two modiﬁcations are often helpful in practice. The ﬁrst is experience replay (Lin, 1992), popularized by Mnih et al. (2015). Instead of using an observed transition to update jus”
Experience Replay for Continual Learning (2018, methodology): “to highly complex tasks. Rehearsing old data via experience replay buffers is a common technique in RL, but such methods have largely been driven by the goal of data-efﬁcient learning on single tasks [7, 16, 20]. Research in this vein has included prioritized replay for maximizing the impact of rare”
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “The commonly used variant in deep reinforcement learning is a kind of hybrid of these two methods, employing a replay buffer (Lin, 1992) and taking gradient steps on the Bellman error objective concurrently with data collection (Mnih et al., 2013).”
Designing Neural Network Architectures using Reinforcement Learning (2016, methodology): “Q -learning, the (cid:15) -greedy strategy and experience replay—in our algorithm design.”
Off-Policy Deep Reinforcement Learning without Exploration (2018, methodology): “Typically, the loss is minimized over mini-batches of tuples of the agent’s past data, (s, a, r, s′) ∈ B, sampled from an experience replay dataset B (Lin, 1992).”
Fitted Q-Learning for Relational Domains (2020, methodology): “For every domain, we used 10% of sampling from the previous history also typically known as experience replay (Lin 1992) in RL literature.”
Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data (2016, methodology): “Experience Replay (ER) (Lin, 1992) (dubbed interleaved learning in McClelland et al (1995)) has been shown to be a successful mechanism in helping online learning algorithms to reuse past experiences.”
Memory Efficient Experience Replay for Streaming Learning (2018, methodology): “s realized updating a DNN with new information often results in catastrophic forgetting of previously learned information [58]. Rehearsal is one of the earliest methods for preventing this phenomenon [50]. Rehearsal, also known as replay and interleaved learning [57], limits catastrophic forgetting ”
Self-Imitation Learning (2018, background): “Experience replay Experience replay (Lin, 1992) is a natural way of exploiting past experiences for parametric policies.”
Continual Normalization: Rethinking Batch Normalization for Online Continual Learning (2022, background): “…(Rusu et al., 2016; Serra et al., 2018; von Oswald et al., 2020; Yoon et al., 2018; Li et al., 2019; Xu & Zhu, 2018), experience replay (Lin, 1992; Chaudhry et al., 2019b; Aljundi et al., 2019a; Riemer et al., 2019; Rolnick et al., 2019; Wu et al., 2019), regularization (Kirkpatrick et…”
Same State, Different Task: Continual Reinforcement Learning without Interference (2021, background): “The associated update does not require on-policy samples, so learning is off-policy using a replay buffer [39], hence ρ is an empirical distribution that represents samples from the buffer.”