Recent Advances in Imitation Learning from Observation

By Faraz Torabi, Garrett Warnell, Peter Stone. 2019

193 citations recorded by the source corpus.

Imitation learning is the process by which one agent tries to learn how to perform a certain task using information generated by another, often more-expert agent performing that same task. Conventionally, the imitator has access to both state and action information generated by an expert performing the task (e.g., the expert may provide a kinesthetic demonstration of object placement using a robotic arm). However, requiring the action information prevents imitation learning from a large number of existing valuable learning resources such as online videos of humans performing tasks. To overcome this issue, the specific problem of imitation from observation (IfO) has recently garnered a great deal of attention, in which the imitator only has access to the state information (e.g., video frames) generated by the expert. In this paper, we provide a literature review of methods developed for IfO, and then point out some open research problems and potential future work.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Leveraging Human Guidance for Deep Reinforcement Learning Tasks (2019, background): “Imitation from observation (IfO) [Torabi et al., 2019c] is the problem of learning directly by observing a trainer performing the task.”
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (2022, background): “There are promising techniques that can potentially overcome this limitation, such as the Learning-from-Observation (LfO) family of algorithms [123, 122, 115, 31].”
Federated Imitation Learning: A Privacy Considered Imitation Learning Framework for Cloud Robotic Systems with Heterogeneous Sensor Data (2019, background): “The expert provides demonstrations of task execution, from which the imitator attempts to mimic the experts behavior [5].”
A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges (2023, unknown): “Subsequent research by Edwards et al. [84] aimed to streamline BCO’s operation by developing a latent forward dynamics model ofﬂine [47].”
Imitation Learning from Video by Leveraging Proprioception (2019, methodology): “Fortunately, the research community has recently begun to focus on addressing the above limitations by considering the speciﬁc problem of imitation from observation ( IfO ) [ Liu et al. , 2018; Torabi et al. , 2019d ] .”
Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos (2022, background): “23 For instance, GAIL 24 constructs an adversarial objective incentivizing the trained policy to exhibit behaviors indistinguishable from those in the target dataset.”
Imitation Learning: A Survey of Learning Methods, Environments and Metrics (2024, unknown): “These new metrics and environments, along with other surveys [1, 34, 35] focusing more on learning techniques, create a lack of standardisation in the evaluation process, making meaningful comparisons between work challenging.”
A Survey of Imitation Learning Methods, Environments and Metrics (2024, methodology): “Hence, we consider the field from a different perspective to the traditional model-free and model-based view employed by Torabi et al. [105].”