Importance of a Search Strategy in Neural Dialogue Modelling

By Ilia Kulikov, Alexander H. Miller, Kyunghyun Cho, Jason Aaron Edward Weston. 2018

44 citations recorded by the source corpus.

Subfields: Natural Language Processing, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Dialogue & Conversational AI

Citations in the corpus

The Curious Case of Neural Text Degeneration (2019, background): “Kulikov et al. (2018) proposed iterative beam search, applied to dialog modeling, which also imposes hard constraints which forces beam hypotheses to be sufficiently different from each other.”
ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons (2019, background): “The integer scores however suffer from differing bias and variance per annotator, which researchers have tried to mitigate (Kulikov et al. 2018), but nevertheless due to its lack of sensitivity often yields comparisons that are not statistically signiﬁcant.”
Neural Text Generation with Unlikelihood Training (2019, background): “Several forms of diverse beam search have been explored (Li et al., 2016; Vijayakumar et al., 2018; Kulikov et al., 2018; Holtzman et al., 2018) which can to some degree decrease the level of repetition of a model by selecting candidates unlike previously chosen ones, due to their variety.”
The Second Conversational Intelligence Challenge (ConvAI2) (2019, methodology): “We attempted to reduce annotator bias in the engagingness scores by using a Bayesian calibration method recently proposed in [8].”
What makes a good conversation? How controllable attributes affect human judgments (2019, methodology): “6 As in the ConvAI2 challenge, each of our 28 model configurations was evaluated by over 100 crowdworkers, and the results were adjusted for annotator variance via a Bayesian calibration (Kulikov et al., 2018).”
Hello, It’s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems (2019, background): “For instance, in task-oriented dialogue modeling the Sequicity model (Lei et al., 2018) sees the classification over the belief state as a generation problem.”
Real or Fake? Learning to Discriminate Machine from Human Generated Text (2019, background): “The approach has been recently re-discovered in the context of dialogue modeling by Kulikov et al. (2018), but here negatives are randomly chosen next utterances from the training dataset.”
Mix-review: Alleviate Forgetting in the Pretrain-Finetune Framework for Neural Language Generation Models (2019, unknown): “To remove bias among annotators, we use the bayesian inference code from Kulikov et al. (2018), and report calibrated mean and standard deviation.”
multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning (2021, background): “…two categories – (1) using improved decoding techniques like beam search with inter-sibling ranking penalty (Li et al., 2016), iterative beam search (Kulikov et al., 2018), diverse beam search (Vijayakumar et al., 2018), and sentence codes (Shu et al., 2019), (2) varying the hidden…”
Beyond User Self-Reported Likert Scale Ratings: A Comparison Model for Automatic Dialog Evaluation (2020, background): “(Kulikov et al., 2018) observe high bias and variance of Likert scores.”
Personalizing Dialogue Agents via Meta-Learning (2019, background): “Several works improved on the initial baselines with various methodologies (Kulikov et al., 2018; Yavuz et al.; Hancock et al., 2019; Lucas et al., 2009; Joshi et al., 2017; Zemlyanskiy and Sha, 2018; Gao et al., 2018).”
Trading Off Diversity and Quality in Natural Language Generation (2020, background): “Vijayakumar et al. (2016), Li & Jurafsky (2016), Tam et al. (2019), and Kulikov et al. (2018) all propose modiﬁcations to beam search to force it to explore a more diverse set of beams.”
Do Massively Pretrained Language Models Make Better Storytellers? (2019, background): “For any probabilistic language model, the generated text is strongly affected by the choice of decoding algorithm – this is especially true for open-ended text generation tasks such as storytelling and chitchat dialogue (Kulikov et al., 2018; Holtz-man et al., 2019).”
Comparison of Diverse Decoding Methods from Conditional Language Models (2019, background): “With these sorts of applications in mind, a variety of alternatives and extensions to beam search have been proposed which seek to produce a set of diverse candidate responses instead of a single high likelihood one (Li et al., 2016a; Vijayakumar et al., 2016; Kulikov et al., 2018; Tam et al., 2019)”
Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue (2020, methodology): “To mitigate annotator bias and inter-annotator variability, we adjust human scoring with Bayesian calibration (Kulikov et al., 2019).”
Learning an Unreferenced Metric for Online Dialogue Evaluation (2020, methodology): “(2019) which are adjusted for annotator variance by using Bayesian calibration (Kulikov et al., 2018) (Table 2).”