Meta-learning autoencoders for few-shot prediction
By Tailin Wu, John Peurifoy, Isaac L. Chuang, Max Tegmark. 2018
24 citations recorded by the source corpus.
Compared to humans, machine learning models generally require significantly more training examples and fail to extrapolate from experience to solve previously unseen challenges. To help close this performance gap, we augment single-task neural networks with a meta-recognition model which learns a succinct model code via its autoencoder structure, using just a few informative examples. The model code is then employed by a meta-generative model to construct parameters for the task-specific model. We demonstrate that for previously unseen tasks, without additional training, this Meta-Learning Autoencoder (MeLA) framework can build models that closely match the true underlying models, with loss significantly lower than given by fine-tuned baseline networks, and performance that compares favorably with state-of-the-art meta-learning algorithms. MeLA also adds the ability to identify influential training examples and predict which additional data will be most valuable to acquire to improve model prediction.
Subfields: Representation & Self-Supervised Learning, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Representation & Self-Supervised Learning
- Razvan Pascanu
- Andrei A. Rusu
- Raia Hadsell
- Dushyant Rao
- Oriol Vinyals
- Jeff Clune
- Yoshua Bengio
- Kenneth O. Stanley
In Meta & Continual Learning
- Razvan Pascanu
- Andrei A. Rusu
- Jeff Clune
- Raia Hadsell
- Kenneth O. Stanley
- Dushyant Rao
- Oriol Vinyals
- Thomas Miconi
Citations in the corpus
- Meta-Learning with Latent Embedding Optimization (2018, background): “Other concurrent works have introduced deep parameter generators (Lacoste et al., 2018; Wu et al., 2018) that can better capture a wider distribution of model parameters, but do not employ gradient-based adaptation.”
- Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity (2018, background): “For example, the weight modifications can themselves be computed by a neural network (Schmidhu-ber, 1993b; Schlag & Schmidhuber, 2017; Munkhdalai & Yu, 2017; Wu et al., 2018).”