Natural Language Generation enhances human decision-making with uncertain information

By Dimitra Gkatzia, Oliver Lemon, Verena Rieser. 2016

62 citations recorded by the source corpus.

Decision-making is often dependent on uncertain data, e.g. data associated with confidence scores or probabilities. We present a comparison of different information presentations for uncertain data and, for the first time, measure their effects on human decision-making. We show that the use of Natural Language Generation (NLG) improves decision-making under uncertainty, compared to state-of-the-art graphical-based representation methods. In a task-based study with 442 adults, we found that presentations using NLG lead to 24% better decision-making on average than the graphical presentations, and to 44% better decision-making when NLG is combined with graphics. We also show that women achieve significantly better results when presented with NLG output (an 87% increase on average compared to graphical presentations).

Subfields: Artificial Intelligence, General Decision Sciences, Social Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Decision Sciences

In Social Psychology

Citations in the corpus

Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, neutral): “Additional pertinent references are: (Blokpoel et al., 2012; Cardona-Rivera, Price, Winer, and Young, 2016; Core, et al., 2006; Gkatzia, Lemon, and Rieser, 2016; Grice, 1975; Rips, Brem, and Bailenson, 1999; Searle, 1985; Slugoski )”
RankME: Reliable Human Ratings for Natural Language Generation (2018, methodology): “In addition, we aim to investigate additional NLG evaluation methods, such as extrinsic task contributions, e.g. (Rieser et al., 2014; Gkatzia et al., 2016).”
Why We Need New Evaluation Metrics for NLG (2017, background): “…context, e.g. (Dušek and Jurčı́ček, 2016); extrinsic evaluation metrics, such as NLG’s contribution to task success, e.g. (Rieser et al., 2014; Gkatzia et al., 2016; Hastie et al., 2016); building discriminative models, e.g. (Hodosh and Hockenmaier, 2016), (Kannan and Vinyals, 2017); or…”
A Survey on the Explainability of Supervised Machine Learning (2020, background): “Gkatzia et al. (2016) found out that human decision-making can be improved by Natural Language Generation (NLG), especially in the area of uncertain data. According to Kass and Finin (1988), the quality of an explanation depends on three different criteria.”
Explanation and Justification in Machine Learning : A Survey Or (2017, methodology): “For example, [Pace and Rosner, 2014] produce explanations of user interactions with a software system, intended for administrators, while [Gkatzia et al. , 2016] explain how a weather forecast was produced and show that it helps readers decide whether or not to believe the forecast.”
No Explainability without Accountability: An Empirical Study of Explanations and Feedback in Interactive ML (2020, background): “Thus, intelligibility research both develops global explanations, such as more transparent models [14, 3, 36, 58] or black-box explanations [37], and local explanations of individual algorithm decisions, which can include input evidence [40, 21], localizations [55, 45], natural language explanations”