Reinforcement Learning in POMDPs with Function Approximation

By Hajime Kimura, K. Miyazaki, Shigenobu Kobayashi. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “However, memories of previous events can help to deal with partially observable Markov decision problems (POMDPs) (e.g., Schmidhuber, 1991c; Ring, 1991, 1993, 1994; Williams, 1992a; Lin, 1993; Teller, 1994; Schmidhuber, 1995; Kaelbling et al., 1995; Littman et al., 1995; Boutilier and Poole, 1996; L”
The Optimal Reward Baseline for Gradient-Based Reinforcement Learning (2001, methodology): “The technique of discounting future rewards was introduced in (Kimura et al., 1995; Kimura et al., 1997), and its effect on reducing variance was noted in (Marbach and Tsitsiklis, 1998).”
Infinite-Horizon Policy-Gradient Estimation (2001, methodology): “Thus the algorithm of Kimura et al. (1997) also estimates thesecond term in the expression forr ( ) given by (19).”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “This triggered work on partially observable Markov decision problems (POMDPs) [223, 222, 227, 204, 205, 206, 316, 148, 278, 122, 152, 25, 114, 160, 126, 308, 309, 183].”
An Analysis of Actor/Critic Algorithms Using Eligibility Traces: Reinforcement Learning with Imperfect Value Function (1998, unknown): “また, = で,かつ criticが出力する割引報酬の推定値V̂ (s)が全ての状態 s について任意の定数 bのとき,図 3のアルゴリズムは確率的傾斜法 [10],[11]と同一になる.”
Variance Reduction Techniques for Gradient Estimates in Reinforcement Learning (2001, background): “Constant baselines, which do not depend on the state or observations, have been widely used [13, 15, 9, 11].”
Approximate Gradient Methods in Policy-Space Optimization of Markov Reward Processes (2003, background): “Finally, the introduction of a discount factor, mostly with the purpose of limiting the variance of the gradient estimates, appears in Jaakkola et al. (1995) and Kimura et al. (1997), as well as in the more recent references (Baxter and Bartlett, 1999; Cao, 2000).”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, unknown): “The papers of Williams (1992), Kimura et al. (1997, 1995), and Baxter and Bartlett (2001) are some of the initial works of research in this area; see also Peshkin (2001) and Ng and Jordan (2000).”
Explainable Reinforcement Learning: A Survey (2020, unknown): “ectly, we have a probability distribution over the possible states instead [37]. For the sake of simplicity, we do not go into further detail and refer the reader to Kaelbling et al. or Kimura et al. [29, 31] for more information. Explainable Reinforcement Learning: A Survey 5 wield the term in a qu”
Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach (2011, unknown): “Finally, another benefit of using policy-gradient algorithms is that these methods react fairly well to learning under noisy dynamics [39].”