Why Attention is Not Explanation: Surgical Intervention and Causal Reasoning about Neural Models

By Christopher Grimsley, Elijah Mayfield, Julia R. S. Bursten. 2020

50 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Caiming Xiong
2. Nazneen Rajani
3. Jesse Vig
4. Richard Socher
5. Ali Madani
6. Yonatan Belinkov
7. Mohit Bansal
8. L. Varshney

Citations in the corpus

• BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, background): “1 We note that all of the above analyses are purely associative and do not attempt to establish a causal link between attention and model behavior (Vig et al., 2020; Grimsley et al., 2020), nor to explain model predictions (Jain & Wallace, 2019; Wiegreffe & Pinter, 2019).”