SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards

By Siddharth Reddy, Anca Dragan, Sergey Levine. 2019

308 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Acme: A Research Framework for Distributed Reinforcement Learning (2020, methodology): “SQIL Discrete & Continuous Actions Soft Q Imitation Learning (Reddy et al., 2020) is a method based on a limit case of AIL, since it consists in using a constant positive reward if the transition comes from the expert demonstrations and a zero reward if the transition comes from the agent interactio”
Feature Expansive Reward Learning: Rethinking Human Input (2020, background): “For instance, the robot could fit an ensemble of functions from traces online, and query for new traces from states where the ensemble disagrees (Reddy et al., 2019).”
Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap (2021, methodology): “Our proposed algorithms bear some resemblance to previously proposed methods, with AdRIL resembling SQIL (Reddy et al. 2019) and AdVIL resembling ValueDICE (Kostrikov et al. 2019).”
Tool Learning with Foundation Models (2023, unknown): “, 2021; Sasaki & Yamashina, 2021), which often requires human annotation, and (2) learning from feedback, which typically involves reinforcement learning (Reddy et al., 2020; Baker et al., 2022).”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, background): “2) Inverse Optimal Control: Traditional IOC algorithms learn an unknown reward function R ( s, a ) from expert demonstrations, where the expert’s reward function can be represented as a linear combination of features [50], [61], [62], [63], [64].”
The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors (2021, background): “These baselines consist of state-of-the-art RL and imitation learning algorithms, including Rain- 7https://preferred.jp/en/ 8https://github.com/minerllabs/baselines 9Preferred Network’s writeup of their experiments using their baselines and the MineRL environments can be found here. bow, SQIL [30], ”
TextGAIL: Generative Adversarial Imitation Learning for Text Generation (2020, methodology): “is often vary large. The original GAIL has difﬁculty to remain stable with such a large action space [17]. We introduce an imitation replay method inspired by the recent imitation learning algorithms [17, 21] to stabilize the training. We ﬁll the experience replay buffer with a ratio of ground truth”
A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges (2023, unknown): “The fundamental idea is to incentivize the agent to consistently remain within the support of the expert policy over time, prompting it to return to the demonstrated states when encountering new states outside the expert’s support [17], [26], [27].”