Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information

By Mario Giulianelli, J. Harding, Florian Mohnert, Dieuwke Hupkes, Willem H. Zuidema. 2018

202 citations recorded by the source corpus.

How do neural language models keep track of number agreement between subject and verb? We show that ‘diagnostic classifiers’, trained to predict number from the internal states of a language model, provide a detailed understanding of how, when, and where this information is represented. Moreover, they give us insight into when and where number information is corrupted in cases where the language model ends up making agreement errors. To demonstrate the causal role played by the representations we find, we then use agreement information to influence the course of the LSTM during the processing of difficult sentences. Results from such an intervention reveal a large increase in the language model’s accuracy. Together, these results show that diagnostic classifiers give us an unrivalled detailed look into the representation of linguistic information in neural models, and demonstrate that this knowledge can be used to improve their performance.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yonatan Belinkov
  2. Mohit Bansal
  3. Percy Liang
  4. Yoav Goldberg
  5. Luke S. Zettlemoyer
  6. Samuel R. Bowman
  7. Yejin Choi
  8. Christopher D. Manning

In Natural Language Processing

  1. Yonatan Belinkov
  2. Samuel R. Bowman
  3. Mohit Bansal
  4. Yoav Goldberg
  5. Tal Linzen
  6. Douwe Kiela
  7. Ellie Pavlick
  8. Percy Liang

In Safety, Robustness & Interpretability

  1. Yonatan Belinkov
  2. Adina Williams
  3. Ellie Pavlick
  4. Samuel R. Bowman
  5. Percy Liang
  6. Douwe Kiela
  7. Sebastian Gehrmann
  8. Isabelle Augenstein

In Computational Linguistics & Semantics

  1. Yonatan Belinkov
  2. Yoav Goldberg
  3. Tal Linzen
  4. James R. Glass
  5. Dieuwke Hupkes
  6. Willem H. Zuidema
  7. Roger Levy
  8. Ivan Titov

Citations in the corpus