Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT

By Zhiyong Wu, Yun Chen, B. Kao, Qun Liu. 2020

216 citations recorded by the source corpus.

By introducing a small set of additional parameters, a probe learns to solve specific linguistic tasks (e.g., dependency parsing) in a supervised manner using feature representations (e.g., contextualized embeddings). The effectiveness of such probing tasks is taken as evidence that the pre-trained model encodes linguistic knowledge. However, this approach of evaluating a language model is undermined by the uncertainty of the amount of knowledge that is learned by the probe itself. Complementary to those works, we propose a parameter-free probing technique for analyzing pre-trained language models (e.g., BERT). Our method does not require direct supervision from the probing tasks, nor do we introduce additional parameters to the probing process. Our experiments on BERT show that syntactic trees recovered from BERT using our method are significantly better than linguistically-uninformed baselines. We further feed the empirically induced dependency structures into a downstream sentiment classification task and find its improvement compatible with or even superior to a human-designed dependency schema.

Subfields: Artificial Intelligence, Large Language Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

Citations in the corpus

Probing Classifiers: Promises, Shortcomings, and Advances (2021, background): “2019) or from distances between word representations, perhaps including perturbations of the input sentence (Wu et al. 2020).”
Pre-Trained Models: Past, Present and Future (2021, background): “Similar ideas have also been explored in ERNIE (Sun et al., 2019c,d) (where a whole entity is masked), NEZHA (Wei et al., 2019), and Whole Word Masking (Cui et al., 2019).”
Conditional probing: measuring usable information beyond a baseline (2021, background): “Other work performs structural analysis of representations without direct supervision (Saphra and Lopez, 2019; Wu et al., 2020).”
Probing Classifiers: Promises, Shortcomings, and Alternatives (2021, background): “These scores may come from attention weights (Raganato and Tiedemann 2018; Clark et al. 2019; Mareˇcek and Rosa 2019; Htut et al. 2019) or from distances between word representations, perhaps including perturbations of the input sentence ( Wu et al. 2020).”
DirectProbe: Studying Representations without Classifiers (2021, background): “Controlled test sets (¸Senel et al., 2018; Jastrzebski et al., 2017) are designed and errors are analyzed to reverse-engineer what information can be encoded by the model (e.g., Marvin and Linzen, 2018; Ravichander et al., 2020; Wu et al., 2020).”
UnNatural Language Inference (2020, methodology): “…inside Transformer LM’s pretrained representations, familiar syntactic structure (Hewitt and Manning, 2019; Jawahar et al., 2019; Lin et al., 2019; Warstadt and Bowman, 2020; Wu et al., 2020), or a familiar order of linguistic operations (Jawahar et al., 2019; Ten-ney et al., 2019), has appeared.”
Explainability for Large Language Models: A Survey (2023, background): “The most straightforward strategy is leave-one-out , which perturbs inputs by removing features at various levels including embedding vectors, hidden units [Li et al. 2017], words [Li et al. 2016], tokens and spans [Wu et al. 2020a] to measure feature importance.”