A Survey of Imitation Learning Methods, Environments and Metrics

By Nathan Gavenski, Felipe Meneguzzi, Michael Luck, Odinaldo Rodrigues. 2024

133 citations recorded by the source corpus.

Imitation learning is an approach in which an agent learns how to execute a task by trying to mimic how one or more teachers perform it. This learning approach offers a compromise between the time it takes to learn a new task and the effort needed to collect teacher samples for the agent. It achieves this by balancing learning from the teacher, who has some information on how to perform the task, and deviating from their examples when necessary, such as states not present in the teacher samples. Consequently, the field of imitation learning has received much attention from researchers in recent years, resulting in many new methods and applications. However, with this increase in published work and past surveys focusing mainly on methodology, a lack of standardisation became more prominent in the field. This non-standardisation is evident in the use of environments, which appear in no more than two works, and evaluation processes, such as qualitative analysis, that have become rare in current literature. In this survey, we systematically review current imitation learning literature and present our findings by (i) classifying imitation learning techniques, environments and metrics by introducing novel taxonomies; (ii) reflecting on main problems from the literature; and (iii) presenting challenges and future directions for researchers.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Social Psychology, Reinforcement Learning, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Social Psychology

In Reinforcement Learning

In Evaluation & Benchmarks

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, background): “[133] fine-tune GPT-3 to answer long-form questions with a web-browsing environment, which allows the model to navigate the web as well as use human feedback to optimize answer quality directly using imitation learning [77].”
Adversarial Training for High-Stakes Reliability (2022, background): “Most empirical work on techniques for aligning powerful AI [12, 13, 14, 15, 16] has focused on achieving good average-case performance in domains where no single action is catastrophic, for example using human trajectory rankings [17, 18, 19] or imitation learning [20, 21].”
Imitating Human Behaviour with Diffusion Models (2023, methodology): “This paper lies under the IL paradigm (Hussein et al., 2017), in particular the line of work that frames IL as matching the state-action occupancy distribution in the dataset, which is related to Inverse RL (Ho and Ermon, 2016; Ghasemipour et al., 2020; Ke et al., 2020).”
Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos (2022, background): “Imitation learning methods 36–39 seek to construct a policy that accurately models the distribution of behavior in some dataset D = { ( o i , a i ) , i 2 { 1 ...”
Reinforcement Learning based Recommender Systems: A Survey (2021, unknown): “In imitation learning, the agent is trained to perform a task from demonstrations [202].”