Submission Category: Reinforcement Learning, Preference: ORAL Approximate Planning in Large POMDPs via Reusable Trajectories

Citations in the corpus

• Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents (2017, background): “Although developed independently by some of this article’s authors, it shares many similarities with the trajectory tree method of Kearns, Mansour, and Ng (1999).”
• Learning from Logged Implicit Exploration Data (2010, background): “For example, in reinforcement learning, the stan dard approach to offline policy evaluation is based on importance weighted samples [3, 11].”
• Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms (2010, positive): “This error bound improves a previous result [17, Theorem 5] for a similar offline evaluation algorithm and similarly provides a sharpened analysis for the T = 1 special case for policy evaluation in reinforcement learning [15].”
• The offset tree for learning with partial labels (2008, background): “Sample complexity results for policy evaluation in reinforcement learning [9] and contextual bandits [13] show that Empirical Risk Minimization type algorithms can find a good policy in a non-interactive setting.”
• Fairness in Reinforcement Learning (2016, background): “[14] Michael Kearns, Yishay Mansour, and Andrew Ng.”
• A Study on Overfitting in Deep Reinforcement Learning (2018, methodology): “With enough computation, algorithms like trajectory tree (Kearns et al., 2000) or Brute (Machado et al.”
• Autonomous helicopter control using reinforcement learning policy search methods (2001, background): “See [7] and [3] for more discussion about policy search and the complexity of uniform bounds on evaluations.”
• Approximate modified policy iteration and its application to the game of Tetris (2015, background): “Using this approach, that is reminiscent of the work by Kearns et al. (2000), the sample complexity of the algorithm no longer depends on the number of iterations.”
• Automating the Construction of Internet Portals with Machine Learning (2000, unknown): “Thus, like in work by Kearns, Mansour, and Ng (2000) , we sample from the state space, and calculate a sum of expected future reward with an explicit roll-out solution using a model.”
• PAC Bounds for Multi-armed Bandit and Markov Decision Processes (2002, background): “To name some of the contributions along this new research direction: E(3) that efficiently computes an -optimal policy [15], sparse sampling [16], restricted class of strategies [13], convergence rate for Q-learning [10], average reward games [7], model based learning [14], and adaptive control of l”
• The Epoch-Greedy algorithm for contextual multi-armed bandits (2007, methodology): “The random trajectories method (Kearns et al., 2000) for learning policies in reinforcement learning with hard horizon T = 1 is essentially the same setting.”
• PAC Reinforcement Learning with Rich Observations (2016, background): “There are some sample complexity guarantees for learning in arbitrarily complex POMDPs, but the bounds we are aware of are quite weak as they scale linearly with |⇧| [14, 19], or require discrete observations from a small set [4].”
• Is Long Horizon Reinforcement Learning More Difficult Than Short Horizon Reinforcement Learning? (2020, background): “Our algorithm has some similarities to the “Trajectory Tree” method in Kearns et al. [2000] in that both attempt to simultaneously evaluate many policies on a “tree” of collected data (using a generative model); a key difference in our approach is that (due to the adaptive nature of data collection)…”