Biases in Predicting the Human Language Model

By Alex B. Fine, Austin F. Frank, T. Florian Jaeger, Benjamin Van Durme. 2014

9 citations recorded by the source corpus.

We consider the prediction of three human behavioral measures ‐ lexical decision, word naming, and picture naming ‐ through the lens of domain bias in language modeling. Contrasting the predictive ability of statistics derived from 6 different corpora, we find intuitive results showing that, e.g., a British corpus overpredicts the speed with which an American will react to the words ward and duke, and that the Google n-grams overpredicts familiarity with technology terms. This study aims to provoke increased consideration of the human language model by NLP practitioners: biases are not limited to differences between corpora (i.e. “train” vs. “test”); they can exist as well between corpora and the intended user of the resultant technology.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Ethical Challenges in Data-Driven Dialogue Systems (2017, background): “…term, which covers a range of problems particularly relevant in natural language systems including: predispositions for regional speech patterns (Fine et al. 2014), discriminatory gender bias (Bolukbasi et al. 2016), personal point-of-view bias (Recasens, Danescu-Niculescu-Mizil, and Jurafsky…”
Efficient Online Scalar Annotation with Bounded Support (2018, background): “E .g., human behavioral measures have been compared with predictability and bias in various corpora (Balota et al., 1999; Fine et al., 2014).”