Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models

By Tongshuang Sherry Wu, Marco Tulio Ribeiro, Jeffrey Heer, Daniel S. Weld. 2021

310 citations recorded by the source corpus.

While counterfactual examples are useful for analysis and training of NLP models, current generation methods either rely on manual labor to create very few counterfactuals, or only instantiate limited types of perturbations such as paraphrases or word substitutions. We present Polyjuice, a general-purpose counterfactual generator that allows for control over perturbation types and locations, trained by finetuning GPT-2 on multiple datasets of paired sentences. We show that Polyjuice produces diverse sets of realistic counterfactuals, which in turn are useful in various distinct applications: improving training and evaluation on three different tasks (with around 70% less annotation effort than manual generation), augmenting state-of-the-art explanation techniques, and supporting systematic counterfactual error analysis by revealing behaviors easily missed by human experts.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Red Teaming Language Models with Language Models (2022, methodology): “…to generate test cases (Dinan et al., 2019; Nie et al., 2020; Ribeiro et al., 2020; Röttger et al., 2021; Xu et al., 2021b; Wallace et al., 2021) and/or detect failures (Dinan et al., 2019; Ziegler et al., 2019; Nie et al., 2020; Stiennon et al., 2020; Xu et al., 2021b; Wu et al., 2021a).”
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2021, methodology): “Generative models have also been used to aid experts with writing contrast sets (Wu et al., 2021; Ross et al., 2021), but to the best of our knowledge, this is the ﬁrst work to investigate the use of generative annotation assistants for crowdwork-ers directly in the annotation loop for NLP. Recent…”
Tailor: Generating and Perturbing Text with Semantic Controls (2021, methodology): “, 2021) or require pairs of original and perturbed sentences (Wu et al., 2021), which are expensive to generalize.”
Explainability for Large Language Models: A Survey (2023, background): “This category encompasses four main streams of approaches for generating explanations including feature attribution-based explanation, attention-based explanation, example-based explanation, and natu-ral language explanation (see Figure 3).”
SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets (2021, background): “Based on evidence that editing is faster than writing [57], this workflow allows us to collect examples more efficiently than through a purely human workflow, while ensuring that the examples are of a higher quality than those produced by a fully automated workflow.”
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook (2024, unknown): “…explanations, such as feature attribution explanation, attention-based explanation, example-based explanation, and natural language explanation [169-172], and global explanations, such as probing-based explanation, neuron activation explanation, concept-based explanation, and mechanistic…”
Post-hoc Interpretability for Neural NLP: A Survey (2021, methodology): “Polyjuice by Wu et al. [134] is primarily a counterfactual dataset generator, and the generation is therefore detached from the model.”
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation (2021, methodology): “A majority of the augmentations that the framework supports are transformations of single sentences that aim to paraphrase these sentences in various ways.”