Language Tasks and Language Games: On Methodology in Current Natural Language Processing Research
By David Schlangen. 2019
21 citations recorded by the source corpus.
"This paper introduces a new task and a new dataset", "we improve the state of the art in X by Y" -- it is rare to find a current natural language processing paper (or AI paper more generally) that does not contain such statements. What is mostly left implicit, however, is the assumption that this necessarily constitutes progress, and what it constitutes progress towards. Here, we make more precise the normally impressionistically used notions of language task and language game and ask how a research programme built on these might make progress towards the goal of modelling general language competence.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Philosophy
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- David Schlangen
- John Yang
- Alexander Wettig
- Ofir Press
- Kexin Pei
- Richard Socher
- Graham Neubig
- Noah A. Smith
In Computer Vision and Pattern Recognition
In Natural Language Processing
- David Schlangen
- John Yang
- Alexander Wettig
- Ofir Press
- Richard Socher
- Graham Neubig
- Marcin Junczys-Dowmunt
- Noah A. Smith
In Philosophy
Citations in the corpus
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (2023, background): “Consequently, a model’s performance on such task conglomerations may not yield actionable, deep insights regarding its capabilities and how to improve them (Schlangen, 2019; Martı́nez-Plumed et al., 2021; Bowman & Dahl, 2021).”
- Targeting the Benchmark: On Methodology in Current Natural Language Processing Research (2020, background): “This figure is from (Schlangen, 2019), of which this is a shorter version developed in a slightly different direction.”