Exponentially Weighted Imitation Learning for Batched Historical Data

By Qing Wang, Jiechao Xiong, Lei Han, Peng Sun, Han Liu, Tong Zhang. 2018

143 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Accelerating Online Reinforcement Learning with Offline Datasets (2020, methodology): “Our method builds on algorithms that implement a maximum likelihood objective for the actor, based on an expectation-maximization formulation of RL [33, 31, 42, 36, 32, 1, 46].”
Offline RL Without Off-Policy Evaluation (2021, positive): “Another idea, proposed by [Wang et al., 2018, Siegel et al., 2020, Wang et al., 2020b, Chen et al., 2020] is to modify an imitation learning algorithm either by ﬁltering or weighting the observed actions to incentivize policy improvement.”
A Minimalist Approach to Offline Reinforcement Learning (2021, methodology): “Other approaches use a weighted version of BC to favor high advantage actions [Wang et al., 2018, Peng et al., 2019, Siegel et al., 2020, Wang et al., 1https://github.com/sfujim/TD3_BC 2020, Nair et al., 2020] or perform BC over an explicit subset of the data [Chen et al., 2020].”
Continuous Doubly Constrained Batch Reinforcement Learning (2021, positive): “From equation (18) in the Proof of Proposition 2 (Appendix A.2) from Wang et al. (2018), we have: J(πφ) ≥ J(πb)− r∗ (1− γ)2 Eπb(s) [ TV ( πφ(a|s), πb(a|s) )] ≥ J(πb)− r∗√ 2(1− γ)2 Eπb(s) √ KL ( πb(a|s), πφ(a|s) ) ≥ J(πb)− r∗√ 2(1− γ)2 √ Eπb(s) [ KL ( πb(a|s), πφ(a|s) )] where the second inequality…”
Critic Regularized Regression (2020, methodology): “Examples of this approach include monotonic advantage re-weighted imitation learning (MARWIL) [38], best-action imitation learning (BAIL) [7], advantage-weighted behavior models (ABM) [31] and advantage weighted regression [28], which has previously been studied in the form of a Fitted Q-iteration…”
Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces (2019, methodology): “Previous work [ Hausknecht and Stone, 2016; Wang et al. , 2018; Wei et al. , 2018b] applied RL to the single-agent version of HFO, letting one agent try to goal without a goal-keeper (1v0).”
Reward-Conditioned Policies (2019, background): “Indeed, a number of recent works have sought to combine imitation learning and reinforcement learning [2, 43, 45].”
Challenges of real-world reinforcement learning: definitions, benchmarks and analysis (2021, background): “Works such as Wang et al. (2018) and Chen et al. (2019b), or Peng et al. (2019) then use the advantage function to select the best actions in the dataset for training behavior cloning.”
A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems (2022, unknown): “sion (AWR) method (analogous to MARWIL [49]).”
LIIR: Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning (2019, background): “In CLDE, actor-critic (AC) style methods [25, 26, 27, 28, 29] are often selected.”
Mastering Complex Control in MOBA Games with Deep Reinforcement Learning (2019, methodology): “We test our method by using the 1v1 mode in Honor of Kings, which is the most popular MOBA game nowadays, and has been actively used as the testbed for recent RL advances (Eisenach et al. 2019; Wang et al. 2018; Jiang, Ekwedike, and Liu 2018).”
BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learning (2019, methodology): “Recent batch DRL algorithms roughly fall into two categories: Q-function-based algorithms such as BCQ [8] and BEAR [16]; and Imitation Learning (IL)-based algorithms such as MARWIL [34] and AWR [23].”