A Diversity-Promoting Objective Function for Neural Conversation Models

By Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, William B. Dolan. 2015

2,767 citations recorded by the source corpus.

Sequence-to-sequence neural network models for generation of conversational responses tend to generate safe, commonplace responses (e.g., "I don't know") regardless of the input. We suggest that the traditional objective function, i.e., the likelihood of output (response) given input (message) is unsuited to response generation tasks. Instead we propose using Maximum Mutual Information (MMI) as the objective function in neural models. Experimental results demonstrate that the proposed MMI models produce more diverse, interesting, and appropriate responses, yielding substantive gains in BLEU scores on two conversational datasets and in human evaluations.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020, background): “Similar to Li et al. [29], Vijayakumar et al. [53] and Massarelli et al. [35], we also investigate the diversity of generations by calculating the ratio of distinct ngrams to total ngrams generated by different models. diverse than RAG-Token generations, and both generate signiﬁcantly more diverse…”
The Curious Case of Neural Text Degeneration (2019, methodology): “(2) Computing the optimum argmax sequence from recurrent neural language models is not tractable, so consider two prominent decoding methods for approximating the argmax: Beam search is the most commonly used approximation in practice (Li et al., 2016c; Shen et al., 2017; Wiseman et al., 2017).”
CoQA: A Conversational Question Answering Challenge (2018, positive): “Considering the results on the test set, the seq2seq model performs the worst, generating frequently occurring answers irrespective of whether these answers appear in the passage or not, a well known behavior of conversational models (Li et al., 2016).”
Sequence-Level Knowledge Distillation (2016, background): “…range of tasks including parsing (Vinyals et al., 2015a), summarization (Rush et al., 2015), dialogue (Vinyals and Le, 2015; Serban et al., 2016; Li et al., 2016), NER/POS-tagging (Gillick et al., 2016), image captioning (Vinyals et al., 2015b; Xu et al., 2015), video generation (Srivastava et…”
LaMDA: Language Models for Dialog Applications (2022, background): “Many automated metrics for dialog models have been studied, including perplexity [16, 17], F1, Hits@1/N [25], USR [46], or BLEU/ROUGE [47, 15, 27].”
Challenges in Detoxifying Language Models (2021, methodology): “We account for this by filtering out degenerate samples with mean distinct-1, distinct-2, distinct-3 score (Li et al., 2015) below 0.5 as done in (Dathathri et al., 2020) before human evaluation.”
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics (2021, background): “…and bigrams (H1, H2), the mean segmented type token ratio over segment lengths of 100 (MSTTR, Johnson, 1944), the ratio of distinct n-grams over the total number of n-grams (Distinct1,2), and the count of n-grams that only appear once across the entire test output (Unique1,2, Li et al., 2016).”
Towards a Human-like Open-Domain Chatbot (2020, methodology): “…typically respond with a generic apology when a human asks something outside their domain; some end-to-end learned chatbots respond “I don’t know” to many inputs (Li et al., 2016a); and Turing Test contest entrants often try to avoid detection by being strategically vague (Venkatesh et al., 2018).”
Pretraining Language Models with Human Preferences (2023, methodology): “We measure diversity in terms of entropy over unigrams expected in a set of N = 2048 LM samples and degeneration in terms of the ratio of all unigrams and distinct unigrams within an average sample (Li et al., 2016).”
ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons (2019, methodology): “The two most used approaches, single-turn pairwise evaluation (Vinyals and Le 2015; Li et al. 2016b), and multi-turn Likert scores (Venkatesh et al. 2017; Zhang et al. 2018; See et al. 2019; Dinan et al. 2019b; Dinan et al. 2019a) have serious limitations.”
Hierarchical Neural Story Generation (2018, methodology): “, 2014) applied to hierarchical story generation are prone to degenerating into language models that pay little attention to the writing prompt (a problem that has been noted in other domains, such as dialogue response generation (Li et al., 2015a)).”
Generating Sentences by Editing Prototypes (2017, background): “Many NLM systems have noted an undesirable tradeoff between grammaticality and diversity, where a temperature low enough to enforce grammaticality results in short and generic utterances (Li et al., 2016).”
A Knowledge-Grounded Neural Conversation Model (2017, methodology): “Following (Sordoni et al. 2015; Li et al. 2016a; Wen et al. 2017), we use BLEU automatic evaluation.”
How Can We Know What Language Models Know? (2019, methodology): “Inspired by the maximum mutual information objective used in Li et al. (2016a), we add the backward log probability logPLM(x|y, tr,i) of each 9In theory, this algorithm can be applied to both masked LMs like BERT and traditional left-to-right LMs, since the masked probability can be computed using…”
Extracting Training Data from Large Language Models (2020, background): “One common failure mode of LMs is their propensity to repeatedly emit the same string over and over [26, 32].”
Learning Through Dialogue Interactions (2016, background): “These include the chit-chat type end-to-end dialogue systems (Vinyals & Le, 2015; Li et al., 2015; Sordoni et al., 2015), which directly generate a response given the previous history of user utterance.”
Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, background): “, 2018) to decode, and these methods are well-known to produce generic text (Li et al., 2015), which makes them susceptible statistical biases present in datasets.”
Does Gender Matter? Towards Fairness in Dialogue Systems (2019, background): “This is consistent with the fact that Seq2Seq generative model tends to produce dull and generic responses [19].”
Plug and Play Language Models: A Simple Approach to Controlled Text Generation (2019, methodology): “Such scores are an indicator of the diversity of the samples generated (Li et al., 2015).”