A Growing Long-term Episodic & Semantic Memory

By Marc Pickett, Rami Al-Rfou, Louis Yuanlong Shao, Chris Tar. 2016

10 citations recorded by the source corpus.

The long-term memory of most connectionist systems lies entirely in the weights of the system. Since the number of weights is typically fixed, this bounds the total amount of knowledge that can be learned and stored. Though this is not normally a problem for a neural network designed for a specific task, such a bound is undesirable for a system that continually learns over an open range of domains. To address this, we describe a lifelong learning system that leverages a fast, though non-differentiable, content-addressable memory which can be exploited to encode both a long history of sequential episodic knowledge and semantic knowledge over many episodes for an unbounded number of domains. This opens the door for investigation into transfer learning, and leveraging prior knowledge that has been learned over a lifetime of experiences to new domains.

Subfields: Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

1. Deva Ramanan
2. Yu-Xiong Wang
3. Martial Hebert
4. Adrian Daniel Popescu
5. Eden Belouadah
6. Ioannis Kanellos

Citations in the corpus

• Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, background): “typically done without dynamically growing the number of parameters across new tasks (as we do). In a broad sense, our approach is related to developmental learning [26, 16, 36] and lifelong learning [41, 25, 39, 29]. Different from the non-parametric shallow models (e.g., nearest neighbors) that in”