Towards Controllable Biases in Language Generation

By Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng. 2020

163 citations recorded by the source corpus.

We present a general approach towards controllable societal biases in natural language generation (NLG). Building upon the idea of adversarial triggers, we develop a method to induce societal biases in generated text when input prompts contain mentions of specific demographic groups. We then analyze two scenarios: 1) inducing negative biases for one demographic and positive biases for another demographic, and 2) equalizing biases between demographics. The former scenario enables us to detect the types of biases present in the model. Specifically, we show the effectiveness of our approach at facilitating bias analysis by finding topics that correspond to demographic inequalities in generated text and comparing the relative effectiveness of inducing biases for different demographics. The second scenario is useful for mitigating biases in downstream applications such as dialogue generation. In our experiments, the mitigation technique proves to be effective at equalizing the amount of biases across demographics while simultaneously generating less negatively biased text overall.

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…(2019); Sweeney and Najafian (2019); Vaidya et al. (2019); Gaut et al. (2020); Gencoglu (2020); Hovy et al. (2020); Hutchinson et al. (2020); Kim et al. (2020); Peng et al. (2020); Rios (2020); Sap et al. (2020); Shah et al. (2020); Sheng et al. (2020); Tan et al. (2020); Zhang et al. (2020a,b).”
Detoxifying Language Models Risks Marginalizing Minority Voices (2021, background): “In fact, other controllable generation techniques, including prompts (Wallace et al., 2019; Sheng et al., 2020; Shin et al., 2020) or conditional LMs (Keskar et al., 2019) will likely exhibit the same type of biases.”
Deduplicating Training Data Makes Language Models Better (2021, background): “This has implications far beyond metrics like per-plexity and validation loss, as learned models re-ﬂect the biases present in their training data (Bender et al., 2021; Wallace et al., 2019; Sheng et al., 2020).”
Societal Biases in Language Generation: Progress and Challenges (2021, methodology): “For autocomplete generation, Sheng et al. (2019, 2020) and Groenwold et al. (2020) compare regard or sentiment scores across demo-graphics, Shwartz et al. (2020) compare names across various intermediate metrics, Vig et al. (2020) measure proportional differences between the amount of bias under a…”
BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation (2021, background): “[32] further showed that adversarial triggers [37] can be used to control biases in language generation.”
Towards Understanding and Mitigating Social Biases in Language Models (2021, methodology): “For global bias, we use a pretrained regard classiﬁer (Sheng et al., 2019; 2020) as well as human judgment.”
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions? (2021, background): “A range of recent works are based on the general idea of models revising their behavior according to changes in their input (Wallace et al., 2019; Gardner et al., 2020; Emelin et al., 2020; Ye and Ren, 2021; Schick and Schütze, 2020; Sheng et al., 2020).”
Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation (2022, background): “Sheng et al. (2020) followed Wallace et al. (2019) and combined different triggers to successfully control the bias direction (i.e., positive, neutral, and negative).”
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts (2021, background): “, 2021), specifically their tendency to generate hateful, offensive, or toxic content (Sheng et al., 2020; Gehman et al., 2020).”
A Distributional Approach to Controlled Text Generation (2020, methodology): “Sheng et al. (2020) introduce a method relying on adversarial triggers (Wallace et al., 2019); this method does not de-bias the whole distribution but only obtains non-biased continuations of given prompts.”
Quark: Controllable Text Generation with Reinforced Unlearning (2022, background): “, via prompts [64, 67] or by enforcing parity across generations [30], could also complement Quark.”
Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities (2021, background): “Formal frameworks for measuring and ensuring fairness (Hendrycks et al. 2020a,b; Sheng et al. 2020) in language are still in nascent stages of development.”
Detect and Perturb: Neutral Rewriting of Biased and Sensitive Text via Gradient-based Decoding (2021, unknown): “Sheng et al. (2020) generate with well-formulated bias triggers based on (Wallace et al., 2019) to equalize biases between demographics.”
Deep Bidirectional Language-Knowledge Graph Pretraining (2022, unknown): “For example, language models have been shown to encode biases about race, gender, and other demographic attributes (Sheng et al. 2020; Weidinger et al. 2021) and generate toxic outputs (Gehman et al. 2020).”
Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts (2021, background): “Previous 1Our code and corpus are available at https:// github.com/abaheti95/ToxiChat research has shown that dialogue models can produce utterances that are gender and racially biased (Wolf et al., 2017; Sheng et al., 2020; Dinan et al., 2020a).”
StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer (2021, background): “Unconditional language models have been shown to perpetuate undesirable stereotypes during generation which disproportionately harm underrepresented social groups (Liang et al., 2020b; Ravfogel et al., 2020; Sheng et al., 2020).”
On-the-Fly Controlled Text Generation with Experts and Anti-Experts (2021, background): “Our study is motivated by the potential harms of using pretrained language models (Bender et al., 2021), speciﬁcally their tendency to generate hate-ful, offensive, or toxic content (Sheng et al., 2020; Gehman et al., 2020).”