Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference

By Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, G. Tesauro. 2018

934 citations recorded by the source corpus.

Lack of performance when it comes to continual learning over non-stationary distributions of data remains a major challenge in scaling neural network learning to more human realistic settings. In this work we propose a new conceptualization of the continual learning problem in terms of a temporally symmetric trade-off between transfer and interference that can be optimized by enforcing gradient alignment across examples. We then propose a new algorithm, Meta-Experience Replay (MER), that directly exploits this view by combining experience replay with optimization based meta-learning. This method learns parameters that make interference based on future gradients less likely and transfer based on future gradients more likely. We conduct experiments across continual lifelong supervised learning benchmarks and non-stationary reinforcement learning environments demonstrating that our approach consistently outperforms recently proposed baselines for continual learning. Our experiments show that the gap between the performance of MER and baseline algorithms grows both as the environment gets more non-stationary and as the fraction of the total experiences stored gets smaller.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Dacheng Tao
  2. Sergey Levine
  3. Yoshua Bengio
  4. Bernt Schiele
  5. Mohit Bansal
  6. Jun Zhu
  7. Han-Jia Ye
  8. Trevor Darrell

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Bernt Schiele
  4. Trevor Darrell
  5. Ming-Hsuan Yang
  6. Xiaogang Wang
  7. Yi Yang
  8. Andrea Vedaldi

In Representation & Self-Supervised Learning

  1. Han-Jia Ye
  2. De-chuan Zhan
  3. Dacheng Tao
  4. Joost van de Weijer
  5. Bernt Schiele
  6. Sergey Levine
  7. Tinne Tuytelaars
  8. Trevor Darrell

In Meta & Continual Learning

  1. Han-Jia Ye
  2. Liyuan Wang
  3. Jun Zhu
  4. De-chuan Zhan
  5. Xingxing Zhang
  6. Jorge Armando Mendez Mendez
  7. Eric Eaton
  8. Joost van de Weijer

Citations in the corpus