Imitation Learning from Video by Leveraging Proprioception

By Faraz Torabi, Garrett Warnell, Peter Stone. 2019

38 citations recorded by the source corpus.

Classically, imitation learning algorithms have been developed for idealized situations, e.g., the demonstrations are often required to be collected in the exact same environment and usually include the demonstrator's actions. Recently, however, the research community has begun to address some of these shortcomings by offering algorithmic solutions that enable imitation learning from observation (IfO), e.g., learning to perform a task from visual demonstrations that may be in a different environment and do not include actions. Motivated by the fact that agents often also have access to their own internal states (i.e., proprioception), we propose and study an IfO algorithm that leverages this information in the policy learning process. The proposed architecture learns policies over proprioceptive state representations and compares the resulting trajectories visually to the demonstration data. We experimentally test the proposed technique on several MuJoCo domains and show that it outperforms other imitation from observation algorithms by a large margin.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Leveraging Human Guidance for Deep Reinforcement Learning Tasks (2019, background): “Imitation from observation (IfO) [Torabi et al. , 2019c] is the problem of learning directly by observing a trainer performing the task.”
A Survey of Imitation Learning Methods, Environments and Metrics (2024, methodology): “Most work use adversarial learning methods to create these reward functions [17, 46, 57, 79, 95–97, 103, 104, 118]; however, some earlier work rely on other inverse reinforcement learning techniques [35, 55, 94].”
Imitation Learning: A Survey of Learning Methods, Environments and Metrics (2024, unknown): “Additionally, Torabi et al. further explore his methods capabilities by applying it to vision tasks [54] by using convolutional neural networks as the discriminator model.”
Recent Advances in Imitation Learning from Observation (2019, background): “To resolve this issue, the authors [Torabi et al. , 2019c; Torabi et al. , 2019b] proposed an algorithm similar to those above, but with the difference that the discriminator considers state transitions , { ( s t , s t +1 ) } , as the input instead of single states.”