Policy-Gradient Algorithms for Partially Observable Markov Decision Processes

By Douglas Aberdeen. 2003

94 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “An important class of DS methods for NNs are Policy Gradient methods (Aberdeen, 2003; Baxter & Bartlett, 2001; Ghavamzadeh & Mahadevan, 2003; Grondman, Busoniu, Lopes, & Babuska, 2012; Grüttner et al., 2010;Heess, Silver, & Teh, 2012; Kohl & Stone, 2004; Peters, 2010; Peters & Schaal, 2008a, 2008b;…”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “1, except that C is now an FNN or RNN trained by evolutionary algorithms [200, 255, 105, 56, 68] applied to NNs [165, 321, 180, 259, 72, 90, 89, 110, 94], or by policy gradient methods [314, 315, 316, 274, 18, 1, 63, 128, 313, 210, 192, 191, 256, 85, 312, 190, 82, 93][245, Sec.”
One Big Net For Everything (2018, background): “6.6], or reinforcement learning [29, 90, 101, 76] through policy gradients [104, 91, 3, 1, 13, 30, 103, 48, 82, 22, 102, 42] [76, Sec.”
Parameter-exploring policy gradients (2010, unknown): “Various approaches have been proposed to reduce this variance Baxter and Bartlett (2000); Aberdeen (2003); Peters and Schaal (2006); Sutton et al. (2000).”
Policy Gradients with Parameter-Based Exploration for Control (2008, background): “Various approaches have been proposed to reduce this variance [6,7,2,8].”
Solving Deep Memory POMDPs with Recurrent Policy Gradients (2007, background): “Work on policy gradient methods with memory has been scarce so far, largely limited to finite state controllers [14, 15].”
Recurrent policy gradients (2010, background): “…these allow for learning policies even with noisy state information [3], work in combination with function approximation [4, 5], are compatible with policies that have internal memory [6], can naturally deal with continuous actions [7–9] and are guaranteed to converge at least to a local minimum.”
Policy Gradient Critics (2007, background): “In spite of their apparent advantages, work on limited-memory stochastic policies has been scarce so far, notable exceptions being finite state-based policy gradients [3, 4] and evolutionary search (e.”
State-Dependent Exploration for Policy Gradient Methods (2008, background): “Neither these nor ideas like baselines [5], the PEGASUS trick [14] or other variance reduction techniques [15] are treated here.”
Natural Actor-Critic (2003, result): “However, similar as in [2], the results presented in this paper might extend to problems with partial state information.”
Exploring parameter space in reinforcement learning (2010, background): “Various approaches have been proposed to reduce this variance [1, 3, 14, 26].”