Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data

By Emily M. Bender, Alexander Koller. 2020

1,362 citations recorded by the source corpus.

The success of the large neural language models on many NLP tasks is exciting. However, we find that these successes sometimes lead to hype in which these models are being described as “understanding” language or capturing “meaning”. In this position paper, we argue that a system trained only on form has a priori no way to learn meaning. In keeping with the ACL 2020 theme of “Taking Stock of Where We’ve Been and Where We’re Going”, we argue that a clear understanding of the distinction between form and meaning will help guide the field towards better science around natural language understanding.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Natural Language Processing

Citations in the corpus

Experience Grounds Language (2020, background): “Moving forward, we encourage a broad re-examination of how NLP frames the relationship between meaning and context (Bender and Koller, 2020) and how pretraining obfuscates our”
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “Evaluation reports should further include improved error analyses, following suggestions by van Miltenburg et al. (2021) and Bender and Koller (2020) who argue for more focus on limitations in addition to aggregated scores.”
Show Your Work: Scratchpads for Intermediate Computation with Language Models (2021, neutral): “The tasks in this paper can be viewed as exploring one criticism of large language models, namely, to what extent do they simply rely on surface-level statistical correlations on text, without learning semantics or world knowledge (Bender & Koller, 2020)?”
AI and the Everything in the Whole Wide World Benchmark (2021, background): “Bender and Koller mention the tendency of machine learning researchers to confuse advancement on certain benchmarks with any tangible evidence of capturing the meaning in language [Bender and Koller, 2020], arguing that benchmarks need to be constructed with care if they are to show evidence of “und”
Finetuned Language Models Are Zero-Shot Learners (2021, methodology): “This work uses language models, for which the risks and potential harms are discussed in Bender & Koller (2020), Brown et al. (2020), Bender et al. (2021), Patterson et al., (2021), and others.”
Program Synthesis with Large Language Models (2021, methodology): “…criticism of language models like the ones we use in this paper is that they learn statistical correlations between tokens without an underlying world-model or mechanism for systematic reasoning, and therefore do not understand the meaning of what they are describing [Bender and Koller, 2020].”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “This lack of control is related to the argument that current NLP systems have a very limited understanding of the social “meaning” of a word or an utterance (Bender & Koller, 2020; Hovy & Yang, 2021).”
Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, background): “There is increasing concern about whether natural language processing systems are learning general linguistic principles, or whether they are simply learning to use surface-level shortcuts (e.g., Bender and Koller, 2020; Linzen, 2020).”
Memory-assisted prompt editing to improve GPT-3 after deployment (2022, background): “Language models are now better than ever before at generating realistic content, but still lack common-sense (Bender and Koller, 2020; Marcus, 2021).”
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, background): “LLMs are not grounded in the physical world and they do not observe the consequences of their generations on any physical process [1].”
AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts (2021, background): “Because LLMs are designed to grasp the form of language, rather than the meaning [7], they can struggle on tasks like sequential arithmetic problems, multi-hop question answering, recognizing and comparing sentences, or those that require branching logic [9, 11, 26, 66, 67].”
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents (2022, background): “While the former set of tasks are limited in their set of linguistic concepts and prove difficult to scale up, the latter tasks usually contain static, noninteractive datasets that lack adequate grounding to extra-linguistic concepts [4].”
What Do NLP Researchers Believe? Results of the NLP Community Metasurvey (2022, background): “We find that the community is nearly evenly split on two controversial issues: whether we should be concerned with AGI (58%, Figure 3c, Q3-1), and whether language models understand language (51%, Figure 3d, Q4-1; Bender and Koller, 2020; Michael, 2020; Potts, 2020; Merrill et al., 2021; Bommasani e”
Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (2022, background): “While PLMs have demonstrated remarkable text generation and understanding capability, they can come with potential risks or harms [2, 3, 5] such as generating misinformation [46] or amplifying harmful biases [49].”
The Turking Test: Can Language Models Understand Instructions? (2020, background): “…it is possible that a larger language model or one trained on instructional language may perform better, it might also be the case, as posited by Bender and Koller (2020), that models may require some form of grounding beyond the linguistic form to learn how to follow instructions and pass the…”
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (2022, background): “Note here that, in contrast to other work (Bender and Koller, 2020; Bisk et al., 2020), we do not focus on whether LLMs “understand” language, instead we examine whether LLMs can answer questions about the emotions and mental states of participants of situations.”
MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound (2022, background): “One of the root causes of such bias is learning objectives that encourage memorization of surface level cooccurences, rather than truly causal factors [55, 9, 117].”
Multimodal Few-Shot Learning with Frozen Language Models (2021, background): “Indeed, philosophers and linguists have questioned whether an un-grounded language model can ever achieve true understanding of the language it processes [5, 2].”
Measuring and Improving Consistency in Pretrained Language Models (2021, background): “Indeed, there is an ongoing discussion about the ability to acquire understanding of “meaning" from raw text signal alone (Bender and Koller, 2020).”