A Framework for Behavioural Cloning

By Michael Bain, C. Sammut. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Recursively Summarizing Books with Human Feedback (2021, methodology): “For example, humans can provide demonstrations of the correct behavior (Bain and Sammut, 1995) or compare two outputs from the model being trained (Christiano et al., 2017), and this data is used to train the model.”
Teaching Autoregressive Language Models Complex Tasks By Demonstration (2021, methodology): “In addition to the work described above, the approach taken in this case study takes inspiration from behavioral cloning [12]–[14], variants of which have been used elsewhere to leverage Transformers for imitation learning [15]–[18].”
Designing Interpretable Approximations to Deep Reinforcement Learning with Soft Decision Trees (2020, methodology): “play and target network in order to stabilize learning. Further details can be obtained in (Mnih et al. 2013). 2.3 Imitation Learning We use the methodology of imitation learning (Abbeel et al. 2007; Bain 1995) to train our simpler and more interpretable decision trees (students) to imitate the comp”
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations (2020, background): “, behavioral cloning, BC [2]) or by inferring an underlying reward function to solve with RL (inverse reinforcement learning, IRL [35]).”
Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control (2021, background): “Typical IL formulations include inverse reinforcement learning (IRL) [30] which infers a cost function from expert demonstrations and behavioral cloning (BC) [1] which directly learns a policy mapping from observations to actions.”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “Learning a policy that directly maps from the state or/and the context to the control input is often referred to as Behavioral Cloning (BC) [Bain and Sammut, 1996].”
Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning (2018, methodology): “Imitation learning has been broadly studied under the twin umbrellas of Behavioral Cloning (BC) (Bain & Sommut, 1999; Ross et al., 2011) and Inverse Reinforcement Learning (IRL) (Ng & Russell, 2000).”
Residual Policy Learning for Shared Autonomy (2020, methodology): “Speciﬁcally, we asked several people ( 9 for Lunar Lander and 14 for Drone Reacher) to perform the task and then trained a surrogate agent for each participant using behavioral cloning [9].”
LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning (2023, unknown): “But during training, we perform behavioral cloning [4] with the following surrogate objective function: where L is a supervised learning loss, e.g., the negative log-likelihood loss, and ⇡ is a Gaussian mixture model.”
Exponentially Weighted Imitation Learning for Batched Historical Data (2018, methodology): “For example, when expert data is available, we can learn a policy directly by predicting the expert action [Bain and Sommut, 1999, Ross et al., 2011].”
Generating synthetic data in finance: opportunities, challenges and pitfalls (2020, methodology): “behavioural cloning [11]) to synthesize this data utilising historical observations and actions taken by an expert”
Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation (2019, methodology): “Training is performed using two separate phases, (1) behavioral cloning (Bain and Sammut, 1999; Wang et al., 2018b; Daftry et al., 2016) and (2) REINFORCE3 policy gradient updates (Williams, 1992).”
Model-based imitation learning by probabilistic trajectory matching (2013, methodology): “In BC, the behavior of a skilled human is recorded and, subsequently, an induction algorithm is executed over the traces of the behavior [6].”
Tool Learning with Foundation Models (2023, unknown): “Behavior cloning (Bain & Sammut, 1995) can be viewed as a simplistic form of imitation learning that focuses on learning policies in a supervised fashion, with the general assumption that the expert’s behavior is optimal or near-optimal.”
Reward is enough (2021, background): “In artificial intelligence, imitation has often been formulated as a problem of learning from demonstration [45] through behavioural cloning [2], where the goal is to reproduce the actions chosen by a teacher, when supplied with explicit data regarding the teacher’s actions, observations and rewards”
Learning action models from plan examples using weighted MAX-SAT (2007, background): “Plan traces have also been exploited to learn situation-action rules in [3], where a decision-tree learning algorithm is applied to learn if-then rules that represent a control strategy for an automated agent.”