Linguistic calibration through metacognition: aligning dialogue agent responses with expected correctness

By Sabrina J. Mielke, Arthur Szlam, Y-Lan Boureau, Emily Dinan. 2020

26 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

You Only Need One Model for Open-domain Question Answering (2021, background): “More importantly however, these language models often suffer from Hallucinations (Shuster et al., 2021; Maynez et al., 2020; Zhou et al., 2021; Zellers et al., 2019; Roller et al., 2021; Mielke et al., 2020), generating factually incorrect but plausiblesounding outputs.”
Retrieval Augmentation Reduces Hallucination in Conversation (2021, background): “, 2020) and studied in depth (Mielke et al., 2020), but so far without resolution.”
Survey of Hallucination in Natural Language Generation (2022, background): “Whereas the first type is described using the term "consistency" [100, 186, 208] or "coherence" [11, 39], people recently start to call the second type "hallucination" [122, 152].”
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems (2022, background): “Mielke et al. (2020) demonstrate that state-of-the-art neural generative chit-chat models frequently respond confidently to questions with incorrect answers.”
Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models (2023, methodology): “One line of research involves training or fine-tuning the LLM to predict the uncertainty [20, 27, 31].”
Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation (2023, background): “Some research has addressed uncertainty or calibration in NLG either by prompting the models to evaluate their own generations or by fine-tuning the generating model to predict its uncertainty (Mielke et al., 2020; Lin et al., 2022a; Kadavath et al., 2022).”
Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation (2021, background): “Mielke et al. (2020) proposed an annotation schema for the correctness and conﬁdence of a model’s response, and calibrated the response such that its correctness aligns with its conﬁdence.”
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? (2022, methodology): “Hallucinations in knowledge-grounded neural dialogue generation is instead a nascent research problem (Mielke et al., 2020; Shuster et al., 2021; Dziri et al., 2021a; Rashkin et al., 2021b 2021a) who introduce frameworks for quantifying attribution in dialogue systems, whereas we conduct a much…”
Understanding and predicting user dissatisfaction in a neural generative chatbot (2021, background): “Though these models have known problems with factual correctness (Mielke et al., 2020), using dialogue history (Sankar et al., 2019), and bias (Dinan et al., 2020), they have nevertheless produced good written conversations when evaluated by crowdworkers or volunteers in carefullycontrolled…”
Automatically Exposing Problems with Neural Dialog Models (2021, methodology): “In the future, we plan to extend our methods to other problems such as generic utterances and hallucination (Mielke et al., 2020).”