Online Linear Regression and Its Application to Model-Based Reinforcement Learning

By Alexander L. Strehl, Michael L. Littman. 2007

100 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Task-Free Continual Learning (2018, neutral): “A ﬁrst set of online learning algorithms consists of different techniques designed to learn a linear model [9, 6, 36, 12].”
Fairness in Learning: Classic and Contextual Bandits (2016, methodology): “Lemma 10 ([Strehl and Littman, 2008]) .”
PAC Optimal Exploration in Continuous Space Markov Decision Processes (2013, background): “For cases where the user can come up with a good set of features but not an exploration strategy, Strehl and Littman (2008) provide an algorithm that can explore in polynomial time in environments whose dynamics can be accurately modeled by linear regression.”
Sample Complexity Bounds of Exploration (2012, methodology): “Furthermore, it can be shown that every PAC-MDP algorithm is probably approximately correct in the average loss criterion (Strehl and Littman, 2008a).”
Reinforcement Learning in Finite MDPs: PAC Analysis (2009, background): “…Efficient learnability in the sample-complexity framework from above implies efficient learnability in a more realistic framework called Average Loss that measures the actual return (sum of rewards) achieved by the agent against the expected return of the optimal policy (Strehl and Littman, 2008b).”
Continual Learning in Neural Networks (2019, background): “A first set of online learning algorithms consists of different techniques designed to learn a linear model [53, 33, 148, 61].”
Interval Estimation for Reinforcement-Learning Algorithms in Continuous-State Domains (2010, background): “A recently proposed framework, KWIK (“Knows What It Knows”), is a formal framework for algorithms that explore efficiently by minimizing the number of times an agent must return the response “I do not know” [23].”
Bounded Optimal Exploration in MDP (2016, methodology): “MBIE and VBE work reasonably if we discard the theoretical guarantee.”
Model-based Reinforcement Learning and the Eluder Dimension (2014, unknown): “Moreover, we present a simple and computationally efficient algorithm posterior sampling for reinforcement learning (PSRL) that satisfies these bounds.”
Knows what it knows: a framework for self-aware learning (2008, background): “It is shown [4] that a noisy d-dimensional linear function can be KWIK-learned with a KWIK bound of Õ(d3), among other polynomial dependencies on 1 and 1 δ .”
Exploring compact reinforcement-learning representations with linear regression (2009, methodology): “One of the ﬁrst uses of the KWIK framework was in the analysis of an online linear regression algorithm used to learn linear transitions in continuous state MDPs [15].”
LQF: Linear Quadratic Fine-Tuning (2020, methodology): “We leave various applications of LQF to future work, including its use in (i) uncertainty quantification, (ii) backward compatibility [44], (iii) leave-one-out cross validation without a validation set [9], (iv) active/online learning [45], (v) counterfactual analysis in the first order, (vi) contin”
Algorithms for Reinforcement Learning (2010, methodology): “TD(λ) solves a model Sutton et al. (2008) and Parr et al.”
CORL: A Continuous-state Offset-dynamics Reinforcement Learner (2008, methodology): “The proof strongly parallels a similar Simulation Lemma in recent work by Strehl and Littman (2008).”
Improved Regret Bounds for Undiscounted Continuous Reinforcement Learning (2015, background): “…results for RL with deterministic transitions in the discounted setting (Bernstein and Shimkin, 2010), as well as for RL with transition functions that are linear in state and action (Strehl and Littman, 2008; Brunskill et al., 2009; Abbasi-Yadkori and Szepesvári, 2011; Ibrahmi et al., 2012).”