S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replay
By Hugo Caselles-Dupré, Michaël Garcia Ortiz, David Filliat. 2019
18 citations recorded by the source corpus.
We consider the problem of building a state representation model for control, in a continual learning setting. As the environment changes, the aim is to efficiently compress the sensory state information without losing past knowledge, and then use Reinforcement Learning on the resulting features for efficient policy learning. To this end, we propose S-TRIGGER, a general method for Continual State Representation Learning applicable to Variational Auto-Encoders and its many variants. The method is based on Generative Replay, i.e. the use of generated samples to maintain past knowledge. It comes along with a statistically sound method for environment change detection, which self-triggers the Generative Replay. Our experiments on VAEs show that S-TRIGGER learns state representations that allows fast and high-performing Reinforcement Learning, while avoiding catastrophic forgetting. The resulting system has a bounded size and is capable of autonomously learning new information without using past data.
Subfields: Artificial Intelligence, Statistical and Nonlinear Physics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Timothée Lesort
- David Filliat
- Natalia Díaz Rodríguez
- Vincenzo Lomonaco
- Davide Maltoni
- Andrei Stoian
- Hugo Caselles-Dupré
- Kalifou René Traoré
In Statistical and Nonlinear Physics
- Brendan McCane
- Anthony V. Robins
- Craig Atkinson
- Lech Szymanski
- Andrei Stoian
- Davide Maltoni
- David Filliat
- Vincenzo Lomonaco
Citations in the corpus
- DisCoRL: Continual Reinforcement Learning via Policy Distillation (2019, methodology): “One possible approach would be to use Continual SRL methods like S-TRIGGER [Caselles-Dupré et al., 2019] or VASE [Achille et al., 2018].”
- Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “Continual learning with replay in GANs has also been used for reinforcement learning (Caselles-Dupré et al., 2019).”
- Continual Learning for Robotics (2019, background): “, 2018a) or reinforcement learning tasks (Caselles-Dupré et al., 2019).”
- Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting (2018, methodology): “Since our work, pseudo-rehearsal has been used to overcome CF in models which have learnt to generate states from previously seen environments [39], [40].”