Automatically Exposing Problems with Neural Dialog Models
By Dian Yu, Kenji Sagae. 2021
9 citations recorded by the source corpus.
Neural dialog models are known to suffer from problems such as generating unsafe and inconsistent responses. Even though these problems are crucial and prevalent, they are mostly manually identified by model designers through interactions. Recently, some research instructs crowdworkers to goad the bots into triggering such problems. However, humans leverage superficial clues such as hate speech, while leaving systematic problems undercover. In this paper, we propose two methods including reinforcement learning to automatically trigger a dialog model into generating problematic responses. We show the effect of our methods in exposing safety and contradiction issues with state-of-the-art dialog models.
Subfields: Natural Language Processing, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Nat McAleese
- Francis Song
- Geoffrey Irving
- Trevor Cai
- Amelia Glaese
- Saffron Huang
- Ethan Perez
- Roman Ring
In Evaluation & Benchmarks
Citations in the corpus
- Red Teaming Language Models with Language Models (2022, background): “, 2019, inter alia) and offensive text from LMs (Wallace et al., 2019; He and Glass, 2019; Liu et al., 2019; Song et al., 2020; Liu et al., 2020; Yu and Sagae, 2021).”