Neuronlike adaptive elements that can solve difficult learning control problems

By Andrew Barto, Richard S. Sutton, C. Anderson. 1983

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Trust Region Policy Optimization (2015, methodology): “To establish a standard baseline, we also included the classic cart-pole balancing problem, based on the formulation from Barto et al. (1983), using a linear policy with six parameters that is easy to optimize with derivative-free black-box optimization methods.”
High-Dimensional Continuous Control Using Generalized Advantage Estimation (2015, methodology): “The cart-pole task was implemented according to the description by Barto et al. (1983). For the cart-pole balancing task, we collected 20 trajectories per batch, whereas we used 50000 timesteps per batch for bipedal locomotion, and 200000 timesteps per batch for quadrupedal locomotion and bipedal st”
Towards Coherent and Cohesive Long-form Text Generation (2018, methodology): “This difference relates to reward with baseline in actor-critic methods (Witten, 1977; Barto et al., 1983; Williams, 1992; Sutton et al., 1999) that typically require a separate critic function as a base-line.”
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018, background): “Actor-critic algorithms are typically derived starting from policy iteration, which alternates between policy evaluation—computing the value function for a policy—and policy improvement—using the value function to obtain a better policy (Barto et al., 1983; Sutton & Barto, 1998).”
Deep learning in neural networks: An overview (2014, methodology): “…or benchmark record-setting Deep Learners actually use one of two supervised techniques: (a) recurrent LSTM (1997) trained by CTC (2006) (Sections 5.13, 5.17, 5.21, 5.22), or (b) feedforward GPU-MPCNNs (2011, Sections 5.19, 5.21, 5.22) based on CNNs (1979, Section 5.4) with MP (1992,…”
Interpretability via Model Extraction (2017, background): “905 cartpole [5] reinforcement learning 100 4 control policy 200.”
Designing Interpretable Approximations to Deep Reinforcement Learning with Soft Decision Trees (2020, background): “A.1 Problem description The cart pole problem is another benchmark problem in reinforcement learning (Barto, Sutton, and Anderson 1983).”
Verifiable Reinforcement Learning via Policy Extraction (2018, background): “For example, in cart-pole [5], we may consider the system to have failed if the pole hit the ground; in this case, all future reward is forfeited, so u = O(T ).”
Programmatically Interpretable Reinforcement Learning (2018, methodology): “We use the OpenAI Gym ‘CartPole-v0’ environment, based on the system presented in (Barto et al., 1983).”
Improving Robot Controller Transparency Through Autonomous Policy Explanation (2017, methodology): “We report results for a parts delivery task (GridWorld [40]) to show application to general planning tasks with discrete state spaces, a stabilization task (CartPole [6]) to show application to control policies with continuous state spaces, and a multiagent parts inspection task in which agents did ”
MOPO: Model-based Offline Policy Optimization (2020, background): “Reinforcement learning algorithms are well-known for their ability to acquire behaviors through online trial-and-error in the environment [3, 65].”
Deep Reinforcement Learning and the Deadly Triad (2018, methodology): “Bootstrapping is also commonly used within policy-gradient and actor-critic methods to learn value functions (Witten, 1977; Barto et al., 1983; Peters and Schaal, 2008; Bhatnagar et al., 2009; van Hasselt, 2012; Degris et al., 2012; Mnih et al., 2016).”
An Off-policy Policy Gradient Theorem Using Emphatic Weightings (2018, methodology): “…methods, a parameterized policy is improved using gradient ascent [Williams, 1992], with seminal work in actor-critic algorithms [Witten, 1977, Barto et al., 1983] and many techniques since proposed to reduce variance of the estimates of this gradient [Konda and Tsitsiklis, 2000, Weaver and…”
A General Family of Robust Stochastic Operators for Reinforcement Learning (2018, neutral): “This problem is first discussed in [4].”
Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks (2016, methodology): “Algorithms that have been used to train deep neural network controllers in reinforcement learning include Q-learning [44, 20], the method of temporal diﬀerences [33, 39], policy gradient and their variants [46, 7], and actor/critic architectures [2, 29, 28].”