Scruples: A Corpus of Community Ethical Judgments on 32, 000 Real-Life Anecdotes

By Nicholas Lourie, Ronan Le Bras, Yejin Choi. 2020

156 citations recorded by the source corpus.

As AI systems become an increasing part of people's everyday lives, it becomes ever more important that they understand people's ethical norms. Motivated by descriptive ethics, a field of study that focuses on people's descriptive judgments rather than theoretical prescriptions on morality, we investigate a novel, data-driven approach to machine ethics. We introduce SCRUPLES, the first large-scale dataset with 625,000 ethical judgments over 32,000 real-life anecdotes. Each anecdote recounts a complex ethical situation, often posing moral dilemmas, paired with a distribution of judgments contributed by the community members. Our dataset presents a major challenge to state-of-the-art neural language models, leaving significant room for improvement. However, when presented with simplified moral situations, the results are considerably more promising, suggesting that neural models can effectively learn simpler ethical building blocks. A key take-away of our empirical analysis is that norms are not always clean-cut; many situations are naturally divisive. We present a new method to estimate the best possible performance on such tasks with inherently diverse label distributions, and explore likelihood functions that separate intrinsic from model uncertainty. Data and code are available at https://github.com/allenai/scruples.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Sociology and Political Science, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Sociology and Political Science

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, background): “The manual collection relies on human annotators to compose new samples or label existing data (Forbes et al., 2020; Hendrycks et al., 2020; Sap et al., 2020; Lourie et al., 2021; Emelin et al., 2021).”
Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits (2023, methodology): “To tackle this misalignment problem, prior work has proposed using binary answers [27, 57], rankings [17, 4], or ratings [70, 39] to model human value preferences.”
On the Machine Learning of Ethical Judgments from Natural Language (2022, background): “…whether the posts’ authors intended to cause offense, whether they contain sexual content, and who the target of the post was; • S CRUPLES (Lourie et al., 2021), a dataset that contains anecdotes and dilemmas, where the dilemmas, used by Jiang et al., consists of natural language…”
Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences (2020, background): “Initial forays into evaluating ethical judgments of NLU models on long-form, un- structured texts were made in (Lourie et al., 2020; Hendrycks et al., 2020), but remained limited to classification.”
Aligning to Social Norms and Values in Interactive Narratives (2022, methodology): “…of everyday ethics, sourced from five existing large-scale datasets, including SOCIAL CHEMISTRY (Forbes et al., 2020), ETHICS Commonsense Morality (Hendrycks et al., 2021a), MORAL STORIES (Emelin et al., 2021), SOCIAL BIAS INFERENCE CORPUS (Sap et al., 2020), and SCRUPLES (Lourie et al., 2021).”
The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems (2022, unknown): “Such meta-data is not present in other popular moral datasets (Hendrycks et al., 2020; Lourie et al., 2021; Forbes et al., 2020; Emelin et al., 2021), but this metadata is critical for understanding the variability of moral intuitions (Talat et al., 2021).”
Whose Opinions Do Language Models Reflect? (2023, background): “…for many tasks—from toxicity detection (Gordon et al., 2021, 2022; Davani et al., 2022; Sap et al., 2022; Goyal et al., 2022), ethics judgements (Lourie et al., 2021), and inference (Pavlick & Kwiatkowski, 2019)—there is inherent variability in what different humans consider the “correct…”