Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment
By Jaap Jumelet, Willem H. Zuidema, Dieuwke Hupkes. 2019
42 citations recorded by the source corpus.
Extensive research has recently shown that recurrent neural language models are able to process a wide range of grammatical phenomena. How these models are able to perform these remarkable feats so well, however, is still an open question. To gain more insight into what information LSTMs base their decisions on, we propose a generalisation of Contextual Decomposition (GCD). In particular, this setup enables us to accurately distil which part of a prediction stems from semantic heuristics, which part truly emanates from syntactic cues and which part arise from the model biases themselves instead. We investigate this technique on tasks pertaining to syntactic agreement and co-reference resolution and discover that the model strongly relies on a default reasoning effect to perform these tasks.
Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yonatan Belinkov
- Yoav Goldberg
- Sebastian Gehrmann
- Percy Liang
- Tal Linzen
- Yejin Choi
- Noah A. Smith
- Hal Daumé Iii
In Natural Language Processing
- Yonatan Belinkov
- Sebastian Gehrmann
- Tal Linzen
- Yoav Goldberg
- Hal Daumé Iii
- Willem H. Zuidema
- Yejin Choi
- Stuart M. Shieber
In Safety, Robustness & Interpretability
- Yonatan Belinkov
- Su Lin Blodgett
- Sebastian Gehrmann
- Willem H. Zuidema
- Tal Linzen
- Hal Daumé Iii
- Oskar van der Wal
- Katrin Schulz
Citations in the corpus
- Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…and Maldonado (2019); Dev and Phillips (2019); Ethayarajh et al. (2019); Gonen and Goldberg (2019); James-Sorenson and Alvarez-Melis (2019); Jumelet et al. (2019); Kaneko and Bollegala (2019); Karve et al. (2019); Kurita et al. (2019); Lauscher and Glavaš (2019); Lauscher et al. (2019);…”
- Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models (2021, positive): “This may be due to plural verbs being encoded as “defaults” in GPT-2, as was found for LSTM LMs in Jumelet et al. (2019).”
- Theories of “Gender” in NLP Bias Research (2022, background): “2 [71] Jaap Jumelet, Willem Zuidema, and Dieuwke Hupkes.”
- The Birth of Bias: A case study on the evolution of gender bias in an English language model (2022, result): “This is in line with earlier work that showed that masculinity acts as the default gender class for a language model (Jumelet et al., 2019).”