Learning with Latent Language

By Jacob Andreas, Daniel Klein, Sergey Levine. 2017

141 citations recorded by the source corpus.

The named concepts and compositional operators present in natural language provide a rich source of information about the abstractions humans use to navigate the world. Can this linguistic background knowledge improve the generality and efficiency of learned classifiers and control policies? This paper aims to show that using the space of natural language strings as a parameter space is an effective way to capture natural task structure. In a pretraining phase, we learn a language interpretation model that transforms inputs (e.g. images) into outputs (e.g. labels) given natural language descriptions. To learn a new concept (e.g. a classifier), we search directly in the space of descriptions to minimize the interpreter’s loss on training examples. Crucially, our models do not require language data to learn these concepts: language is used only in pretraining to impose structure on subsequent learning. Results on image classification, text editing, and reinforcement learning show that, in all settings, models with a linguistic parameterization outperform those without.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Learning Transferable Visual Models From Natural Language Supervision (2021, methodology): “As another alternative, search could be performed at inference time over many natural language explanations of a given image, similar to approach proposed in Learning with Latent Language Andreas et al. (2017).”
Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “, 2020), and even help in image and reinforcement learning settings (Andreas et al., 2018).”
ALICE: Active Learning with Contrastive Natural Language Explanations (2020, background): “training loop is effective in various settings (Andreas et al., 2018; Mu et al., 2020).”
ExpBERT: Representation Engineering with Natural Language Explanations (2020, methodology): “Others have also used language to directly produce parameters of a classifier (Ba et al., 2015) and as part of the parameter space of a classifier (Andreas et al., 2017).”
Zero-Shot Text-to-Image Generation (2021, neutral): “This process can be seen as a kind of language-guided search (Andreas et al., 2017), and is also similar to the auxiliary text-image matching loss proposed by Xu et al.”
Scalable agent alignment via reward modeling: a research direction (2018, background): “Natural language has the potential to achieve better generalization if the latent space is represented using language (Andreas et al., 2018) and possibly generalize in a way that is more predictable to humans.”
Can language models learn from explanations in context? (2022, background): “…tuning models for computer vision (Hendricks et al., 2016; Schramowski et al., 2020; Majumder et al., 2021; Mu et al., 2020), for mathematical problem solving by program induction (Ling et al., 2017), or relational reasoning and reinforcement learning (Andreas et al., 2018; Lampinen et al., 2021).”
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data (2021, methodology): “Other works map directly from text to parameters in models (Ba et al., 2015; Andreas et al., 2018), in effect learning a prior p(θ|text) (though Andreas et al.”
Pre-Trained Language Models for Interactive Decision-Making (2022, background): “Semi-Supervised Skill Learning with Latent Language (SL) 3 [38] shows that LMs can serve as an effective backbone for hierarchical policies that express plans as natural language strings [2, 4].”
Pretrained Transformers as Universal Computation Engines (2021, background): “Language might also be a natural choice of model for this blackboard, as there are hypotheses that language may serve as a good multipurpose high-level representation for cognitive behavior and conscious planning (Andreas et al., 2017; Goyal & Bengio, 2020).”
A Survey of Reinforcement Learning Informed by Natural Language (2019, methodology): “[ Andreas et al. , 2018 ] explore the idea of natural language descriptions as a policy parametrization in a 2D navigation task adapted from [ Janner et al. , 2018 ] .”
Skill Induction and Planning with Latent Language (2021, background): “Language-based supervision and representation The use of natural language annotations to scaffold learning, especially in computer vision and program synthesis applications, has been the subject of a number of previous studies (Branavan et al., 2009; Frome et al., 2013; Andreas et al., 2018; Wong et”
NILE : Natural Language Inference with Faithful Natural Language Explanations (2020, background): “The idea of using natural language strings as a latent space has been explored to capture compositional task structure (Andreas et al., 2018).”
Language as an Abstraction for Hierarchical Deep Reinforcement Learning (2019, background): “Andreas et al. [2] leverage language descriptions to rapidly adapt to unseen environments through structured policy search in the space of language; each environment is described by one sentence.”
Learning by Abstraction: The Neural State Machine (2019, background): “In our view, these results point to the strongest quality of our approach. several prior works have argued for the great potential of abstractions and compositionality in enhancing models of deep learning [8, 10].”
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training (2022, background): “It dates back to at least Andreas et al. (2018) and saw renewed interest in the past few months due to the prevalence of large PLMs. Andreas et al. (2018) and Vong and Lake (2022) learn natural language descriptions that function as few-shot classiﬁers within an image-text matching model.”
DALL-E: CREATING IMAGES FROM TEXT (2021, background): “Likelihood goals tend to make modelling short-range dependence among pixels a priority [2], therefore most of modelling is needed to collect high frequency data rather than a low frequency structure which makes things visibly recognizable.”
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models (2023, unknown): “Language grounding has been studied extensively both in terms of intelligent agents [19–22] and of robotics [23, 6, 24, 25, 5, 7, 26], where language can be used as a tool for compositional goal specification [5, 27–33], semantic anchor for training multi-modal representation [12, 34, 35], or as an…”
State-Denoised Recurrent Neural Networks (2018, background): “Turning to research with a cognitive science focus, Andreas et al. [2017] have proposed a model that efﬁciently learns new concepts and control policies by operating in a linguistically constrained representational space.”