Human Language Understanding & Reasoning

By Christopher D. Manning. 2022

Abstract The last decade has yielded dramatic and quite surprising breakthroughs in natural language processing through the use of simple artificial neural network computations, replicated on a very large scale and trained over exceedingly large amounts of data. The resulting pretrained language models, such as BERT and GPT-3, have provided a powerful universal language understanding and generation base, which can easily be adapted to many understanding, writing, and reasoning tasks. These models show the first inklings of a more general form of artificial intelligence, which may lead to powerful foundation models in domains of sensory experience beyond just language.

Subfields: Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Large Language Models

1. Bo Li
2. Kevin Chen-Chuan Chang
3. Bowen Zhou
4. Jie Huang
5. Alessandro Lenci
6. Peng Qi

In Prompting & Reasoning

1. Kevin Chen-Chuan Chang
2. Jie Huang
3. Alessandro Lenci

Citations in the corpus

• Emergent Abilities of Large Language Models (2022, neutral): “Better than random and not correlated with scale: boolean expressions, crash blossom, dynamic counting, entailed polarity hindi, epistemic reasoning, factuality of summary, fantasy reasoning, gender sensitivity chinese, gender sensitivity english, high low game, identify math theorems, intersect geo”
• Understanding Natural Language Understanding Systems. A Critical Analysis (2023, methodology): “Improvements are also to be expected by refining the pretraining methods, as well as their adaptability to new tasks (Yogatama et al. 2019, Bommasani et al. 2021, Manning 2022).”
• ChatGPT and large language models in academia: opportunities and challenges (2023, methodology): “ChatGPT is powered by a LLM, a type of deep learning model that emerged around 2018 [2].”
• Towards Reasoning in Large Language Models: A Survey (2022, background): “In fact, there is also an ongoing debate about whether language models can actually understand language or capture meaning (Bender and Koller, 2020; Li et al., 2021; Manning, 2022; Piantasodi and Hill, 2022).”
• Trustworthy AI: From Principles to Practices (2021, background): “They not only show the potential to a more general form of AI [234], but also bring new challenges and opportunities to the establishment of trustworthy AI.”