Why Gender and Age Prediction from Tweets is Hard: Lessons from a Crowdsourcing Experiment

By Dong Nguyen, Dolf Trieschnigg, A. Seza Doğruöz, Rilana Gravel, Mariët Theune, Theo Meder, Franciska de Jong. 2014

135 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, methodology): “Further, a dataset might just not capture suﬃcient information to make an inference; for example, there are limits to approaches for predicting users’ demographics solely based on the messages they post in social media (Nguyen et al., 2014).”
Analyzing Biases in Human Perception of User Age and Gender from Text (2016, background): “The majority vote accuracy on predicting the gender of Twitter users is 85.8% with the majority class baseline being 51.9% female, a result comparable to a previous study (Nguyen et al., 2014).”
Zooming in on Gender Differences in Social Media (2016, background): “Using a large dataset of over 350 words, we show that gender-based word disambiguation is possible, and that there are indeed differences between the ways certain words are used by men and women.”
Bleaching Text: Abstract Features for Cross-lingual Gender Prediction (2018, background): “Flekova et al. (2016) found that people tend to rely too much on stereotypical lexical indicators, while Nguyen et al. (2014) show that more than 10% of the Twitter users do actually not employ words that the crowd associates with their biological sex.”
Returning the N to NLP: Towards Contextually Personalized Classification Models (2020, background): “…treating demographic variables as ﬂuid and social, e.g. modeling what inﬂuences speakers to show more or less of their identity through language, or jointly modeling variation between and within speakers (Eckert and McConnell-Ginet, 2013; Nguyen et al., 2014; Bamman et al., 2014; Eisenstein, 2013).”
Obfuscating Gender in Social Media Writing (2016, background): “Even though this binary may not be an accurate reﬂection of the gender performance of users on social media (Bamman et al., 2014; Nguyen et al., 2014), we operate under the presumption that most demographic classiﬁers also use two genders.”
Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning (2019, background): “Such demographic biases can also pose issues to data relying on subjective judgments of a subgroup [23, 26] in tasks such as sentiment analysis and content moderation.”
Semantic Variation in Online Communities of Practice (2018, unknown): “…a substantial amount of work has used NLP techniques to study correlations between linguistic variables and macro-sociological categories such as age (Nguyen et al., 2013), gender (Nguyen et al., 2014; Burger et al., 2011; Ciot et al., 2013), and other demographic factors (Eisenstein et al., 2014).”
Confounds and Consequences in Geotagged Twitter Data (2015, background): “Other work in the domain of demographic prediction often involves more complex methods (Nguyen et al., 2014; Volkova and Durme, 2015), but since it is not the focus of our research, we take a relatively simple approach here, assuming no labeled data for demographic attributes.”
Gender Prediction for Chinese Social Media Data (2017, background): “, 2011), in addition to the fact that many users produce language atypical of their demographic (Bamman et al., 2014; Nguyen et al., 2014).”
TweetGenie: Development, Evaluation, and Lessons Learned (2014, methodology): “The data collected proved to be valuable: using the data, we reﬂected on the task of inferring gender and age from tweets and the limitations of current systems (Nguyen et al., 2014).”
Studying User Income through Language, Behaviour and Affect in Social Media (2015, background): “Crowdsourcing perceived attribute annotations—collecting the subjective impressions workers get when reading the content of user profiles—is not trivial [28, 29].”
ConStance: Modeling Annotation Contexts to Improve Stance Classification (2017, background): “Still, researchers often decide what context to show annotators without rigorously evaluating how their decisions will affect annotations, on tasks from gender identiﬁcation to political leanings (Chen et al., 2015; Nguyen et al., 2014; Burger et al., 2011; Cohen and Ruths, 2013).”
Computational Sociolinguistics: A Survey (2016, methodology): “This issue is discussed in depth in some recent computational work related to gender, specifically Bamman, Eisenstein, and Schnoebelen (2014) and Nguyen et al. (2014) who provide a critical reflection on the operationalization of gender in CL studies.”