Beyond Accuracy: Behavioral Testing of NLP Models with CheckList

By Marco Tulio Ribeiro, Tongshuang Sherry Wu, Carlos Guestrin, Sameer Singh. 2020

1,473 citations recorded by the source corpus.

Although measuring held-out accuracy has been the primary approach to evaluate generalization, it often overestimates the performance of NLP models, while alternative approaches for evaluating models either focus on individual tasks or on specific behaviors. Inspired by principles of behavioral testing in software engineering, we introduce CheckList, a task-agnostic methodology for testing NLP models. CheckList includes a matrix of general linguistic capabilities and test types that facilitate comprehensive test ideation, as well as a software tool to generate a large and diverse number of test cases quickly. We illustrate the utility of CheckList with tests for three tasks, identifying critical failures in both commercial and state-of-art models. In a user study, a team responsible for a commercial sentiment analysis model found new and actionable bugs in an extensively tested model. In another user study, NLP practitioners with CheckList created twice as many tests, and found almost three times as many bugs as users without it.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, methodology): “formal evaluation processes (Mitchell et al., 2019; Ribeiro et al., 2020), we propose releasing evalu-”
Are NLP Models really able to Solve Simple Math Word Problems? (2021, methodology): “We create new problems by applying certain variations to existing problems, similar to the work of Ribeiro et al. (2020).”
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics (2021, background): “For example, we may improve the training sets, make the test sets more challenging, or probe for specific skills a model must exhibit with testonly datasets (Perez-Beltrachini and Gardent, 2017; Linzen, 2020; Ribeiro et al., 2020; Schlegel et al., 2020).”
HateCheck: Functional Tests for Hate Speech Detection Models (2020, methodology): “To generate test cases at scale, we use templates (Dixon et al., 2018; Ribeiro et al., 2020), in which we replace tokens for protected group identifiers (e.”
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (2022, background): “Some of these efforts augment humans (through guidelines, templates, programmatic generation of attacks, and various combinations thereof) to devise test cases that cause systems to fail [45, 46, 29, 21, 30, 55, 6, 23].”
Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, background): “…for example, failure to satisfy intuitive invariances, such as invariance to typographical errors or seemingly irrelevant word substitutions (Ribeiro et al., 2020); ambiguity in measuring progress in language understanding (Zellers et al., 2019); and reliance on stereotypical associations…”
Red Teaming Language Models with Language Models (2022, methodology): “Prior work relies on human annotators to generate test cases (Dinan et al., 2019; Nie et al., 2020; Ribeiro et al., 2020; Röttger et al., 2021; Xu et al., 2021b; Wallace et al., 2021) and/or detect failures (Dinan et al.”
RuleBERT: Teaching Soft Rules to Pre-Trained Language Models (2021, methodology): “To test the symmetry relationship between predicates, we use the SRL test in CheckList (Ribeiro et al., 2020), which focuses on behavioral testing of NLP models; we use its test set for the duplicate-question detection task (QQP) (Wang et al.”
Measuring Association Between Labels and Free-Text Rationales (2020, background): “, 2019) and checklists (Ribeiro et al., 2020), are conceptually similar to our measurements, but study different model properties.”
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022, background): “This leads to models that seem to achieve human-level performance on in-domain test sets, yet are brittle when given out-of-domain or adversarial examples (Ribeiro et al., 2020; Glockner et al., 2018).”
Tailor: Generating and Perturbing Text with Semantic Controls (2021, methodology): “…related are the creation of minimally edited datasets, either through manual rewriting (Gard-ner et al., 2020; Kaushik et al., 2020), or creating perturbation templates (Andreas, 2020; Li et al., 2020; Ribeiro et al., 2020; Wu et al., 2019); T ailor reduces the human e ﬀ orts these studies require.”
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models (2021, methodology): “We also evaluate model capabilities with CheckList (Ribeiro et al., 2020) for Sentiment and QQP. Reported model performances are averaged across multiple data samples and random seeds (Appendix B.2).”
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021, methodology): “HATECHECK is a suite of functional tests for hate speech detection models, based on the testing framework introduced by Ribeiro et al. (2020).”
Measuring and Improving Consistency in Pretrained Language Models (2021, background): “Ribeiro et al. (2020) use consistency for more robust evaluation.”
Dynabench: Rethinking Benchmarking in NLP (2021, methodology): “Recent work has also focused on evaluating model diagnostics through counterfactual augmentation (Kaushik et al., 2020), decision boundary analysis (Gardner et al., 2020; Swayamdipta et al., 2020), and behavioural testing (Ribeiro et al., 2020).”
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021, background): “…for state-of-the-art results in NLP. How- ever, these approaches are heavily dependent on large-scale labeled datasets that are expensive to create, and the resulting models still generalize poorly to out-of-distribution inputs created with small, harmless perturbations (Ribeiro et al., 2020).”
Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definitions (2020, background): “…learning models, their intended uses, and potential abuses (Bender and Friedman, 2018; Gebru et al., 2018; Mitchell et al., 2019; Pineau, 2020; Ribeiro et al., 2020), aimed at making “critical information accessible that previously could only be found by users with great effort” (Bender and…”