A Reinforcement Learning Method for Maximizing Undiscounted Rewards

By Anton Schwartz. 1993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “…& Atkeson, 1993; Morimoto & Doya, 2000; Peng & Williams, 1996; Prokhorov &Wunsch, 1997; Rummery &Niranjan, 1994; Santamaría, Sutton, & Ram, 1997; Schwartz, 1993; Singh, 1994; Sutton & Barto, 1998; Sutton, Szepesvári, & Maei, 2008; Tsitsiklis & van Roy, 1996; van Hasselt, 2012; Watkins, 1989;…”
Reinforcement learning in robotics: A survey (2013, background): “The off-policy variant is called R-learning (Schwartz, 1993), which is closely related to Q-learning, with the updates”
Reinforcement Learning: A Survey (1996, neutral): “Schwartz (1993) examined the problem of adapting Q-learning to an average-reward framework.”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “The parameters or weights of C are trained to maximize reward by a standard RL method such as Q-learning or similar methods [17, 292, 293, 172, 254, 212, 262, 10, 122, 188, 157, 281, 26, 216, 197, 272, 311, 9, 163, 174, 22, 27, 2, 137, 276, 156, 284].”
Using continuous action spaces to solve discrete problems (2009, methodology): “This algorithm uses a one step temporal difference update to update its approximation of the value of the state-action pair , [9] is similar to that of Q-Iearning: The main difference lies in the lack of a max operator in the case of Sarsa.”
A Reinforcement Learning Algorithm Based on Policy Iteration for Average Reward: Empirical Results with Yield Management and Convergence Analysis (2004, methodology): “Some of the pioneering work in this area can be attributed to Schwartz (1993) and Mahadevan (1994, 1996a, 1996b).”
Consequentialist conditional cooperation in social dilemmas with imperfect information (2017, methodology): “Note that while we will prove results in the undiscounted setting, for δ sufficiently close to 1, the optimal policy is the same in the undiscounted and discounted setting, so standard discounted policy gradient techniques can still be used (Schwartz, 1993).”
Average Reward Reinforcement Learning: Foundations, Algorithms, and Empirical Results (2005, methodology): “Average adjusted values satisfy a nice linearity property [31] (assuming a deterministic MDP): V y V x = r.”
Reinforcement Learning based Recommender Systems: A Survey (2021, background): “RL-based Methods RLRS Year SR PO RF EB Metrics Dataset Application* WebWatcher [83] 1997 SR1 Q-learning R1 Oline Accuracy CMU Web Preda et al. [84] 2005 SR1 Sarsa(�) R1 Online RDR N/A Web MDP [50] 2005 SR1 Policy iteration R2 Online RC, ED Mitos E-commerce Taghipour et al. [85] 2007 SR1 Q-learning R”
Temporal-Logic-Based Reward Shaping for Continuing Reinforcement Learning Tasks (2020, background): “R-learning (Schwartz 1993; Mahadevan 1996) is a classical approach for average-reward RL but lacks theoretical properties.”
Learning Algorithms for Markov Decision Processes with Average Cost (2001, methodology): “Although Schwartz presents some intuitive arguments to justify his algorithm along with some numerical results, he does not provide any convergence analysis.”
Reinforcement Learning: an Overview (2000, methodology): “The average-reward model An algorithm of reinforcement learning, the R-Learning [11, 26], uses a third criteria, that takes the average of future reinforcements: R(t) = lim n→∞ 1 n n ∑”