Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design

By A. S. Bergman, Gavin Abercrombie, Shannon L. Spruit, Dirk Hovy, Emily Dinan, Y-Lan Boureau, Verena Rieser. 2022

22 citations recorded by the source corpus.

Over the last several years, end-to-end neural conversational agents have vastly improved their ability to carry unrestricted, open-domain conversations with humans. However, these models are often trained on large datasets from the Internet and, as a result, may learn undesirable behaviours from this data, such as toxic or otherwise harmful language. Thus, researchers must wrestle with how and when to release these models. In this paper, we survey recent and related work to highlight tensions between values, potential positive impact, and potential harms. We also provide a framework to support practitioners in deciding whether and how to release these models, following the tenets of value-sensitive design.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, background): “Deng et al. (2023) proposes a taxonomic framework to tackle these issues, and Bergman et al. (2022) delves into the balance between potential positive and negative impacts from releasing dialogue models.”
The Gradient of Generative AI Release: Methods and Considerations (2023, background): “Ultimately personal values around openness are a large factor in decision making [102], and tensions can be further examined by output modality [9].”
Regulating ChatGPT and other Large Generative AI Models (2023, background): “Already now, LGAIMs’ unbridled capacities may be harnessed to take manipulation, fake news, and harmful speech to an entirely new level [7-11].”
A Watermark for Large Language Models (2023, unknown): “As these systems become more pervasive, there is increasing risk that they may be used for malicious purposes (Bergman et al., 2022; Mirsky et al., 2023).”