How Reliable are Model Diagnostics?

By Vamsi Krishna Aribandi, Yi Tay, Donald Metzler. 2021

21 citations recorded by the source corpus.

In the pursuit of a deeper understanding of a model's behaviour, there is recent impetus for developing suites of probes aimed at diagnosing models beyond simple metrics like accuracy or BLEU. This paper takes a step back and asks an important and timely question: how reliable are these diagnostics in providing insight into models and training setups? We critically examine three recent diagnostic tests for pre-trained language models, and find that likelihood-based and representation-based model diagnostics are not yet as reliable as previously assumed. Based on our empirical findings, we also formulate recommendations for practitioners and researchers.

Subfields: Evaluation & Benchmarks, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

1. Siva Reddy
2. Nicholas Meade

In Safety, Robustness & Interpretability

1. Nicholas Meade
2. Elinor Poole-Dayan
3. Siva Reddy

Citations in the corpus

• An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models (2021, background): “Furthermore, Aribandi et al. (2021) demonstrated the relative instability of the performance of pre-trained language models, such as BERT, on CrowS-Pairs (and StereoSet) across different pre-training runs.”