Delphi: Towards Machine Ethics and Norms

By Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Maxwell Forbes, Jon Borchardt, Jenny T Liang, Oren Etzioni, Maarten Sap, Yejin Choi. 2021

135 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, background): “Jiang et al. (2021) proposed several ethical question-answering tasks, involving the judgment of the ethics of various actions.”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “These existing solutions tackle content-related harms, but future work in reducing discrimination and toxicity in LMs may also benefit from KBs that encode social (Chang et al., 2020), cultural, (Hershcovich et al., 2022), and moral norms (Hendrycks et al., 2021; Jiang et al., 2021).”
A General Language Assistant as a Laboratory for Alignment (2021, methodology): “The work with the most similar approach to PMP for alignment was the very recent Delphi [JHB + 21], which trains a general-purpose ethical critic.”
Memory-assisted prompt editing to improve GPT-3 after deployment (2022, methodology): “We use a subset 3 of the dataset provided by DEL - PHI (Jiang et al., 2021).”
Red Teaming Language Models with Language Models (2022, background): “A potential solution is to learn a classifier that detects many harms, as in Askell et al. (2021); Jiang et al. (2021), to generalize to novel harms.”
What Does it Mean for a Language Model to Preserve Privacy? (2022, background): “Another related line of work is context-aware, long-form, ethical and persona-based, response generation [62, 60, 43, 55], where the chatbot or dialog system is supposed to hold a conversation with previous context taken into consideration [50].”
Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study (2023, background): “(2022) claim that the Delphi language model (Jiang et al., 2021), designed for moral prediction, necessarily has an inconsistent model of normative ethics.”
PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change (2022, background): “In this paper, we are particularly interested in the line of research efforts that investigate (and showcase) the reasoning capabilities of Large Language models–including commonsense reasoning [31, 27, 5], logical reasoning [29], and even ethical reasoning [14].”
Challenges and Strategies in Cross-Cultural NLP (2022, background): “Grounding these objectives explicitly in ethical values and norms, and acknowledging that these may differ across cultures, is essential for cross-cultural NLP. Jiang et al. (2021), for example, introduce C OMMONSENSE N ORM B ANK and Delphi, a moral reasoning resource and model, respectively, reﬂect”
On the Planning Abilities of Large Language Models - A Critical Investigation (2023, background): “Of particular interest to us in this paper is the thread of efforts that aim to investigate (and showcase) reasoning abilities of LLMs–including commonsense reasoning [35, 29, 7], logical reasoning [33], and even ethical reasoning [15].”
UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training (2022, background): “The past few years has seen a lot of activity toward unified model designs (Friedman et al., 2021; Aghajanyan et al., 2021; Gupta et al., 2021; Tafjord and Clark, 2021; Lourie et al., 2021; Jiang et al., 2021; Aribandi et al., 2021; Zhong et al., 2021a; Chen et al., 2021; Zhong et al., 2022).”
Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits (2023, methodology): “To tackle this misalignment problem, prior work has proposed using binary answers [27, 57], rankings [17, 4], or ratings [70, 39] to model human value preferences.”
ProsocialDialog: A Prosocial Backbone for Conversational Agents (2022, methodology): “Delphi is a generative model demonstrating great performance on language-based commonsense moral reasoning, trained on 1.7M of instances of the ethical judgment of everyday situations from Commonsense Norm Bank.”
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies (2022, methodology): “Project Delphi introduced the Commonsense Norm Bank (Jiang et al., 2021) of over 1.7 million human moral judgments.”
On the Machine Learning of Ethical Judgments from Natural Language (2022, methodology): “Works like Jiang et al. (2021) and Hendrycks et al. (2021) go beyond this in fine-tuning language models to output moral prescriptions for sentential descriptions of situations.”
The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems (2022, unknown): “Most recently, Jiang et al. (2021) combined the S OCIAL -”