Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?

By Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, Kai-Wei Chang. 2021

59 citations recorded by the source corpus.

Is it possible to use natural language to intervene in a model's behavior and alter its prediction in a desired way? We investigate the effectiveness of natural language interventions for reading-comprehension systems, studying this in the context of social stereotypes. Specifically, we propose a new language understanding task, Linguistic Ethical Interventions (LEI), where the goal is to amend a question-answering (QA) model's unethical behavior by communicating context-specific principles of ethics and equity to it. To this end, we build upon recent methods for quantifying a system's social stereotypes, augmenting them with different kinds of ethical interventions and the desired model behavior under such interventions. Our zero-shot evaluation finds that even today's powerful neural language models are extremely poor ethical-advice takers, that is, they respond surprisingly little to ethical interventions even though these interventions are stated as simple sentences. Few-shot learning improves model behavior but remains far from the desired outcome, especially when evaluated for various types of generalization. Our new task thus poses a novel language understanding challenge for the community.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Alignment & RLHF

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Alignment & RLHF

Citations in the corpus

The Capacity for Moral Self-Correction in Large Language Models (2023, background): “Our hypothesis is not entirely new (see §2 for related work, especially [51, 63]) but we believe our experiments and results are.”
Towards conversational diagnostic artificial intelligence (2025, unknown): “The development of strategies for alignment 77 , self-improvement 78–81 and scalable oversight mechanisms 82 has enabled the large-scale deployment of such conversational systems in the real world 16,83 .”
In-BoXBART: Get Instructions into Biomedical Multi-Task Learning (2022, background): “Furthermore, many works focused on investigating whether LMs understands meaning of natural language and prompts (Webson and Pavlick, 2021; Zhao et al., 2021).”
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks (2022, background): “…Nadeem et al., 2021) and expressed in specific tasks (Nangia et al., 2020; Kirk et al., 2021; Schramowski et al., 2022; Prabhumoye et al., 2021; Srinivasan and Bisk, 2021; Kirk et al., 2021; Parrish et al., 2021; Baldini et al., 2022; Czarnowska et al., 2021; Dev et al., 2021a; Zhao et al., 2021).”
Prompting GPT-3 To Be Reliable (2022, unknown): “Takeaway ( i ) Few-shot prompting of GPT-3 is more robust than supervised models such as ﬁne-tuned BERT and RoBERTa, under all three settings (domain shift, perturbations, spurious correlation).”