Linguistic I Ssues in L Anguage Technology Lilt on Achieving and Evaluating Language-independence in Nlp on Achieving and Evaluating Language-independence in Nlp

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

How Can We Accelerate Progress Towards Human-like Linguistic Generalization? (2020, background): “A model can be sample-efficient for English, but not for other languages.”
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics (2021, background): “It is often assumed that work on English transfers to other languages (Bender, 2011).”
Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science (2018, background): “The NLP community has recognized critical limitations in the ﬁeld’s existing data catalog, leaving many language communities un-derserved (Bender, 2011; Munro, 2015; Jurgens et al., 2017).”
The State and Fate of Linguistic Diversity and Inclusion in the NLP World (2020, background): “You might also be aware of the fact that most of our NLP systems, which are typically declared language agnostic, are not truly so (Bender, 2011).”
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models (2022, background): “We note all benchmarks in Table 1 are designed to measure harms in English, indicating a lack of linguistic and cultural diversity that is well documented across other language tasks [51, 9, 10, 23].”
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning (2020, background): “Such a narrow scope not only curbs the development of natural language understanding tools in other languages (Bender, 2011; Ponti et al., 2019a), but also exacerbates the Anglo-centric bias in modeling commonsense reasoning.”
Taxonomy of Risks posed by Language Models (2022, background): “Choices on model architecture may also have an impact, as it has been proposed that the architecture of current LMs and particularly tokenisation is well-suited to English, but not morphologically more complex languages [14, 79, 162].”
Many Languages, One Parser (2016, background): “Theoretically, the study of linguistic typology tells us that many languages share morphological, phonological, and syntactic phenomena (Bender, 2011); therefore, the mainstream approach misses an opportunity to exploit relevant supervision from typologically related languages.”
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages (2022, background): “…or a few high-resource Indo-European languages (Elliott et al., 2016), disregarding the enormous diversity of the world’s languages and cultures (Bender, 2011; Ponti et al., 2019; Liu et al., 2021), which exacerbated the Anglo-centric bias (Jauhar et al., 2018; Ponti et al., 2020) of V&L…”
The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants (2023, background): “Many modeling techniques are being presented as language-agnostic but have only truly been evaluated in a small number of languages (Bender, 2011), risking”
Data Statements for NLP: Toward Mitigating System Bias and Enabling Better Science (2018, background): “The NLP community has recognized critical limitations in the field’s existing data catalog, leaving many language communities underserved (Bender, 2011; Munro, 2015; Jurgens et al., 2017).”
Writing System and Speaker Metadata for 2,800+ Language Varieties (2022, methodology): “Language diversity in NLP Our work is closely aligned with the goal of increasing language diversity and representation in NLP research, which has recently received renewed attention (Bender, 2011; Joshi et al., 2020).”
Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold (2022, background): “pared to languages with relatively free word order (Bender, 2011).”
From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers (2020, background): “, 2012; Fort, 2016), coupled with plethora of structurally different NLP tasks, lead to existence of such datasets only for a handful of resource-rich languages (Bender, 2011; Ponti et al., 2019; Joshi et al., 2020).”