Generalized polynomial approximations in Markovian decision processes

By P. Schweitzer, A. Seidmann. 1985

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Investigating Contingency Awareness Using Atari 2600 Games (2012, methodology): “While automatic feature construction methods (Parr et al. 2007; Konidaris, Osentoski, and Thomas 2011) and generic feature sets (Schweitzer and Seidmann 1985) have been proposed, current techniques do not explicitly incorporate any notion of contingency awareness.”
The Arcade Learning Environment: An Evaluation Platform for General Agents (2012, methodology): “Such methods include polynomial bases (Schweitzer & Seidmann, 1985), sparse distributed memories (Kanerva, 1988) and locality sensitive hashing (LSH) (Gionis et al., 1999).”
Surprising Negative Results for Generative Adversarial Tree Search (2018, methodology): “Since in RL we do not have access internal expectation, we instead minimize the Bellman residual [43][28, 3] which has been extensively used in DRL literature [36, 47]:”
Efficient Solution Algorithms for Factored MDPs (2003, methodology): “Approximate linear programming for MDPs using linear value function approximation was introduced by Schweitzer and Seidmann (1985), although the approach was somewhat deprecated until fairly recently due the lack of compelling error analyses and the lack of an effective method for handling the…”
Efficient Exploration Through Bayesian Deep Q-Networks (2018, methodology): “In value based model free RL, the core of most prominent approaches is to learn the Q-function through minimizing the Bellman residual (Schweitzer & Seidmann, 1985; Lagoudakis & Parr, 2003; Antos et al., 2008) h i and temporal difference (TD) update (Tesauro, 1995) where the tuple ( x, a, r, x ′ )…”
Finite-sample Analysis of Bellman Residual Minimization (2010, unknown): “In particular, we focus on the Bellman residual minimization approach (Schweitzer and Seidmann, 1985; Baird, 1995; Munos, 2003; Lagoudakis and Parr, 2003; Scherrer, 2010) when a generative model is available, that is, for any state-action pair it is possible to obtain the immediate reward and an ind”
Context-specific multiagent coordination and planning with factored MDPs (2002, methodology): “The linear programming approach can be adapted to use this value function representation [12] by changing the objective function to a ̄ Aw, and changing the constraints to have the form, Aw _> T*Aw.”
Policy evaluation with temporal differences: a survey and comparison (2015, methodology): “Bellman residual minimization (BRM) was one of the first approaches for approximate policy evaluation proposed by Schweitzer and Seidmann (1985). It calculates the least-squares solution of the MSBE given by”
Relaxing dynamic programming (2006, methodology): “Also, [10] and [15] present methods where the approximate value function is parameterized as a linear combination of a set of basis functions.”
Algorithm-Directed Exploration for Model-Based Reinforcement Learning in Factored MDPs (2002, background): “(An analogous linear program was proposed for the discounted infinite horizon case in [23].)”
Least-Squares Policy Iteration (2003, unknown): “∆μ to be a diagonal matrix with entries μ(s, a), the weighted least-squares solution of the system would be: w = ( (Φ− γPΠπΦ) T∆μ(Φ− γPΠπΦ) )−1 (Φ− γPΠπΦ) T∆μR , The Bellman residual minimizing approach has been proposed (Schweitzer and Seidmann, 1985) as a means of computing approximate state value”
A unifying framework for computational reinforcement learning theory (2009, background): “The need for exploration also exists in other machine-learning settings such as active learning, in which a supervised-learning agent decides what input–output data to use in its training set; see Settles [2009] for a recent survey.”
SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation (2017, methodology): “These saddle point problems are derived from the Lagrangian dual of the linear programming formulation of Bellman equations (Schweitzer & Seidmann, 1985; de Farias & Van Roy, 2003).”
Approximate Dynamic Programming for Ambulance Redeployment (2010, methodology): “The linear programming approach for ADP finds a good set of values for the parameters by solving a large linear program whose decision variables are {r p : p = 1, . . . , P }; see Schweitzer and Seidmann (1985), de Farias and Van Roy (2003) and Adelman and Mersereau (2007).”
Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error (2022, background): “The Bellman equation has played a prominent role in many historically significant approaches (Schweitzer & Seidmann, 1985; Baird, 1995; Bradtke & Barto, 1996; Antos et al., 2008; Sutton et al., 2009), as well as the more modern family of deep RL algorithms (Mnih et al., 2015; Lillicrap et al.,…”
Faster saddle-point optimization for solving large-scale Markov decision processes (2019, background): “In fact, relaxing the linear program underlying (1) through the introduction of the feature map F for approximating the value function v ∗ is one of the oldest ideas in approximate dynamic programming, originally introduced by Schweitzer and Seidman (1985).”