TX-Ray: Quantifying and Explaining Model-Knowledge Transfer in (Un-)Supervised NLP

By Nils Rethmeier, Vageesh Kumar Saxena, Isabelle Augenstein. 2019

23 citations recorded by the source corpus.

Subfields: Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Jonathan Berant
2. Omer Levy
3. Mor Geva
4. Roei Schuster
5. Yejin Choi
6. Barbara Plank
7. Mohit Bansal
8. Percy Liang

In Representation & Self-Supervised Learning

1. Alan Ramponi
2. Roei Schuster
3. Jonathan Berant
4. Barbara Plank
5. Mor Geva
6. Omer Levy
7. Yoav Goldberg
8. Ido Dagan

Citations in the corpus

• Transformer Feed-Forward Layers Are Key-Value Memories (2020, methodology): “Considerable attention has been given to demystifying the operation of neural NLP models.”
• Neural Unsupervised Domain Adaptation in NLP—A Survey (2020, background): “This can be linked to the question on what representations capture (Belinkov and Glass, 2019) and how knowledge transfers (Rethmeier et al., 2020).”
• Analyzing Individual Neurons in Pre-trained Language Models (2020, unknown): “…Dalvi et al., 2019; Suau et al., 2020) and have other potential benefits such as controlling bias and manipulating system’s behaviour (Bau et al., 2019), model distillation and compression (Rethmeier et al., 2020), efficient feature selection (Dalvi et al., 2020), and guiding architectural search.”