Better Conversations by Modeling, Filtering, and Optimizing for Coherence and Diversity

By Xinnuo Xu, Ondrej Dusek, Ioannis Konstas, Verena Rieser. 2018

66 citations recorded by the source corpus.

We present three enhancements to existing encoder-decoder models for open-domain conversational agents, aimed at effectively modeling coherence and promoting output diversity: (1) We introduce a measure of coherence as the GloVe embedding similarity between the dialogue context and the generated response, (2) we filter our training corpora based on the measure of coherence to obtain topically coherent and lexically diverse context-response pairs, (3) we then train a response generator using a conditional variational autoencoder model that incorporates the measure of coherence as a latent variable and uses a context gate to guarantee topical consistency with the context and promote lexical diversity. Experiments on the OpenSubtitles corpus show a substantial improvement over competitive neural models in terms of BLEU score as well as metrics of coherence and diversity.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, methodology): “Gehman et al. (2020a) attempt to apply these techniques to toxicity in LLMs. Control techniques have also been employed in dialog, for example, to control for style (Smith et al., 2020b), engagingness (See et al., 2019), or coherence (Xu et al., 2018).”
History for Visual Dialog: Do we really need it? (2020, methodology): “While the dialog community approaches these phenomena by e.g. learning better models of coherence (Xu et al., 2018), we believe that evaluation metrics also need to be improved for this task, as widely discussed for other generation tasks, e.g. (Liu et al., 2016; Novikova et al., 2017; Reiter,…”
Identifying Untrustworthy Samples: Data Filtering for Open-domain Dialogues with Bayesian Optimization (2021, methodology): “[53] introduced Coherence with cosine similarity to measure the semantic relationship between contexts and responses, and then filtered out pairs with low coherence score.”
Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach (2021, methodology): “[63], we measure specificity using word embedding similarity between seeker post and rewritten response post (using embeddings from BERT [14]).”
Linguistically-Informed Specificity and Semantic Plausibility for Dialogue Generation (2019, methodology): “During testing, we condition on the highest level of our similarity metric in order to generate maximally coherent responses (Xu et al., 2018b).”
DynaEval: Unifying Turn and Dialogue Level Evaluation (2021, methodology): “CoSim is a common method for dialogue coherence assessment (Xu et al., 2018; Zhang et al., 2018a).”