Twitter Universal Dependency Parsing for African-American and Mainstream American English

By Su Lin Blodgett, Johnny Tian-Zheng Wei, Brendan T. O'Connor. 2018

76 citations recorded by the source corpus.

Due to the presence of both Twitter-specific conventions and non-standard and dialectal language, Twitter presents a significant parsing challenge to current dependency parsing tools. We broaden English dependency parsing to handle social media English, particularly social media African-American English (AAE), by developing and annotating a new dataset of 500 tweets, 250 of which are in AAE, within the Universal Dependencies 2.0 framework. We describe our standards for handling Twitter- and AAE-specific features and evaluate a variety of cross-domain strategies for improving parsing with no, or very little, in-domain labeled data, including a new data synthesis approach. We analyze these methods’ impact on performance disparities between AAE and Mainstream American English tweets, and assess parsing accuracy for specific AAE lexical and syntactic features. Our annotated data and a parsing model are available at: http://slanglab.cs.umass.edu/TwitterAAE/.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “Other representational harms Jørgensen et al. (2015); Hovy and Søgaard (2015); Blodgett et al. (2016); Blodgett and O’Connor (2017); Blodgett et al. (2018); Curry and Rieser (2018); Dixon et al. (2018); Park et al. (2018); Thelwall (2018); Webster et al. (2018); Badjatiya et al. (2019); Bagdasaryan…”
Differential Privacy Has Disparate Impact on Model Accuracy (2019, background): “This task involves classifying Twitter posts from the recently proposed corpus of African-American English [4, 5] as positive or negative.”
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models (2022, methodology): “TwitterAAE and SAE/AAVE Pairs both compare a Benchmark Demographic Groups RTP [40] None TwitterAAE [15] Speakers of AAE SAE/AAVE Pairs [44] Speakers of AAVE Winogender [87] Gender Winobias [108] Gender Gender & Occ [21, 85] Gender Deconfounding [43] Gender, Profession, Country TruthfulQA [74] None…”
A Survey of Race, Racism, and Anti-Racism in NLP (2021, background): “For example: • Annotation schema Returning to Blodgett et al. (2018), this work deﬁnes new parsing standards for formalisms common in AAE, demonstrating how parsing labels themselves were not designed for racialized language varieties.”
Detect and Perturb: Neutral Rewriting of Biased and Sensitive Text via Gradient-based Decoding (2021, unknown): “Example 1 shows an excerpt of a tweet written by an African-American revealed by vocabulary usage (future tense of gone → “is going to”) (Blodgett et al., 2018).”
Dialect Diversity in Text Summarization on Twitter (2020, background): “[9] Su Lin Blodgett, Johnny Wei, and Brendan O’Connor.”
Exploring the Role of Grammar and Word Choice in Bias Toward African American English (AAE) in Hate Speech Classification (2022, methodology): “’s AAE twitter dataset [3] as our AAE data.”
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021, background): “, 2019; Saunders and Byrne, 2020), 14 (Blodgett et al., 2018), 15 (Bamman et al.”