Incorporating Dialectal Variability for Socially Equitable Language Identification

By David Jurgens, Yulia Tsvetkov, Dan Jurafsky. 2017

104 citations recorded by the source corpus.

Language identification (LID) is a critical first step for processing multilingual text. Yet most LID systems are not designed to handle the linguistic diversity of global platforms like Twitter, where local dialects and rampant code-switching lead language classifiers to systematically miss minority dialect speakers and multilingual speakers. We propose a new dataset and a character-based sequence-to-sequence model for LID designed to support dialectal and multilingual language varieties. Our model achieves state-of-the-art performance on multiple LID benchmarks. Furthermore, in a case study using Twitter for health tracking, our method substantially increases the availability of texts written by underrepresented populations, enabling the development of “socially inclusive” NLP tools.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “Questionable correlations Jurgens et al. (2017); Madnani et al. (2017); Rudinger et al. (2017); Zhao et al. (2017); Burns et al. (2018); Díaz et al. (2018); Kiritchenko and Mohammad (2018); Lu et al. (2018); Rudinger et al. (2018); Shen et al. (2018); Bordia and Bowman (2019); Cao and Daumé (2019);…”
Measuring Bias in Contextualized Word Representations (2019, background): “6 Related Work NLP applications ranging from core tasks such as coreference resolution (Rudinger et al., 2018) and language identiﬁcation (Jurgens et al., 2017), to downstream systems such as automated essay scoring (Amorim et al., 2018), exhibit inherent social biases which are attributed to the…”
Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science (2018, background): “The NLP community has recognized critical limitations in the field’s existing data catalog, leaving many language communities underserved (Bender, 2011; Munro, 2015; Jurgens et al., 2017).”
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems (2018, background): “This paper does not examine the differences in accuracies of systems on text produced by different races or genders, as was done by Hovy (2015); Blodgett et al. (2016); Jurgens et al. (2017); Buo-lamwini and Gebru (2018).”
Fairness Without Demographics in Repeated Loss Minimization (2018, background): “Jurgens et al., 2017), dependency parsing (Blodgett et al., 2016), part-of-speech tagging (Hovy & Sgaard, 2015), academic recommender systems (Sapiezynski et al., 2017), and automatic video captioning (Tatman, 2017).”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, background): “The same effect was reported for language identiﬁcation difﬁculties for African-American Vernacular English (Blodgett and O’Connor, 2017; Jurgens et al., 2017).”
Demoting Racial Bias in Hate Speech Detection (2020, background): “One thread of work focuses on removing bias at the data level, through reducing annotator bias (Sap et al., 2019) and augmenting imbalanced datasets (Jurgens et al., 2017).”
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization (2019, background): “Such models are problematic when they violate equity considerations (Jurgens et al., 2017; Buolamwini & Gebru, 2018) or rely on spurious correlations : misleading heuristics that work for most training examples but do not always hold.”
Distributionally Robust Neural Networks (2020, background): “Such models are problematic when they violate equity considerations (Jurgens et al., 2017; Buolamwini & Gebru, 2018) or rely on spurious correlations: misleading heuristics that work for most training examples but do not hold on different test distributions.”
How to Measure Gender Bias in Machine Translation: Optimal Translators, Multiple Reference Points (2020, background): “…algorithms ( Chen et al., 2018; Dastin, 2018; Schwarm, 2018), ageist and racist ad targeting (Angwin et al., 2017; Barocas and Selbst, 2016; Chen et al., 2018), and the lack of using regional dialects in the training corpus of Natural Language Processing (NLP) algorithms (Jurgens et al., 2017).”
Equity Beyond Bias in Language Technologies for Education (2019, background): “Accounting for dialects and other language variation has been moderately more successful, with examples in speech recognition (Kraljic et al., 2008), parsing (Gimpel et al., 2011), and classiﬁcation (Jurgens et al., 2017).”
Twitter Universal Dependency Parsing for African-American and Mainstream American English (2018, background): “Expanding the linguistic coverage of NLP tools to include minority and col-loquial dialects would help support equitable language analysis across sociolinguistic communities, which could help information retrieval, translation, or opinion analysis applications (Jurgens et al., 2017).”
A Fast, Compact, Accurate Model for Language Identification of Codemixed Text (2018, background): “Langid.py (Lui and Baldwin, 2012), CLD2 10 and fastText (Joulin et al., 2016, 2017)—all are popular off-the-shelf tools for monolingual language ID. Sentence-level predictions for EquiLID and LanideNN models are obtained by simple voting.”
Data Statements for NLP: Toward Mitigating System Bias and Enabling Better Science (2018, background): “This scenario is inspired by Jurgens et al. (2017), who a similar case study to motivate training language ID systems on more representative datasets.”
Quantifying Social Biases in Contextual Word Representations (2019, background): “, 2018) and language identification (Jurgens et al., 2017), to downstream systems such as automated essay scoring (Amorim et al.”