Human-level concept learning through probabilistic program induction

By Brenden M. Lake, Ruslan Salakhutdinov, Josh Tenenbaum. 2015

3,332 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets (2016, methodology): “[17] have been able to learn representations using probabilistic inference over Bayesian programs, which achieved convincing one-shot learning results on the OMNI dataset.”
Meta-Learning for Low-Resource Neural Machine Translation (2018, background): “One of the most successful applications of meta-learning has been on few-shot (or one-shot) learning (Lake et al., 2015), where a neural network is trained to readily learn to classify inputs based on only one or a few training examples.”
Do Better ImageNet Models Transfer Better? (2018, methodology): “This problem is closely related to few-shot learning [40, 74, 58, 64, 21, 64, 51], but these methods are typically evaluated with training and test classes from the same distribution.”
Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, background): “Transfer learning is related to both multi-task learning [32, 4, 28, 10, 11, 45, 24, 2] and learning novel categories from few examples [49, 19, 21, 35, 51, 22, 5, 13, 50, 47, 31].”
Scaling Laws for Transfer (2021, background): “[LST15] showed few-shot learning for generating handwritten characters with probabilistic program induction inline with human capabilities of few-shot learning on such a task.”
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models (2021, background): “Few-shot learning—the ability to learn tasks with limited examples—is an important academic and practical challenge (Lake et al., 2015).”
Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, background): “of images from few examples [34, 40]. What accounts for this monumental difference in data-efﬁciency between biological and machine vision? While highlystructured representations (e.g. as proposed by [33]) may improve data-efﬁciency, it remains unclear how to program explicit structures that capture”
Environmental drivers of systematicity and generalization in a situated agent (2019, background): “It is also worth noting that none of our experiments reﬂect the human ability to learn (rather than extend existing knowledge) quickly (as in, e.g. Lake et al. (2015); Botvinick et al. (2019)).”
Joint Concept Learning and Semantic Parsing from Natural Language Explanations (2017, methodology): “• BoW: A logistic regression (LR) classifier over bag-of-words representation of emails • BoW tf-idf: LR classifier over bag-of-words representation, with tf-idf weighting • Para2Vec: LR classifier over a distributed representation of documents, using deep neural network approach by Le and Mikolov (”
What can the brain teach us about building artificial intelligence? (2016, background): “ For example, one could test whether the visual representation of characters in high-level visual regions reflects the similarity predicted by the generative model of character perception proposed by Lake et al. (2015). The current advances in AI reinvigorate the interaction between cognitive scienc”
Towards Conceptual Compression (2016, methodology): “The recently introduced Omniglot dataset (Lake et al., 2015) is comprised of 1628 character classes drawn from multiple alphabets with just 20 samples per class.”
VideoGPT: Video Generation using VQ-VAE and Transformers (2021, neutral): “Finally, building generative models of the world around us is considered as one way to measure our understanding of physical common sense and predictive intelligence (Lake et al., 2015).”
Calibrate Before Use: Improving Few-Shot Performance of Language Models (2021, background): “Few-shot learning—the ability to learn tasks with limited examples—is an important aspect of intelligence (Lake et al., 2015; Yogatama et al., 2019).”
Language Model Cascades (2022, background): “Recent advances in program synthesis suggest the possibility of probabilistic program induction (Lake et al., 2015; Goodman et al., 2014) to search for cascades which solve a target task, rather than assuming a ﬁxed probabilistic program structure.”
Learning multiple visual domains with residual adapters (2017, background): “Omniglot [20] consists of 1623 different handwritten characters from 50 different alphabets.”
CrossTransformers: spatially-aware few-shot transfer (2020, background): “Few-shot learning [25, 35, 46, 53] has recently been primarily addressed in the meta-learning framework [57, 71, 80], where a model learns an update rule for the parameters of a base-learner model [6, 7, 72] through a sequence of training episodes [79, 88].”
Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes (2019, methodology): “This general approach relates to methods for meta-learning [1, 2] and few-shot learning [3].”
Meta-learning with differentiable closed-form solvers (2018, methodology): “In this work, we evaluated our proposed methods R2-D2 and LR-D2 in the few-shot learning scenario (Fei-Fei et al., 2006; Lake et al., 2015; Vinyals et al., 2016; Ravi & Larochelle, 2017; Hariharan & Girshick, 2017), which consists in learning how to discriminate between images given one or very few…”