Towards developing general models of usability with PARADISE

By Marilyn A. Walker, Candace A. Kamm, Diane J. Litman. Natural Language Engineering, 2000

334 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/5940c8bfd16728c5f9bb66e807f78f43dc7a38e4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Human-Computer Interaction

Citations in the corpus

Topic-based Evaluation for Conversational Bots (2018, neutral): “Previous work has introduced evaluation metrics related to task success for evaluating task-based bots [22].”
Natural Language Generation as Planning Under Uncertainty for Spoken Dialogue Systems (2009, background): “For comparison: (Walker et al., 2000) report on R between .”
Automating spoken dialogue management design using machine learning: An industry perspective (2008, background): “…the effect of increasing automation rate, as evidenced for example in the strong correlation found between certain features of the interaction (e.g. the number of speech recognition errors), the user experience, and the overall automation rate in a PARADISE analysis (Walker et al., 2000, 2001).”
Automatic Learning and Evaluation of User-Centered Objective Functions for Dialogue System Optimisation (2008, methodology): “First, we measure the predictive power of our models within the same data set using 10-fold cross validation, and across the different systems by testing models trained on one sys-tem to predict perceived Task Ease for another system, following a method introduced by (Walker et al., 2000).”
Natural Language Generation as Incremental Planning Under Uncertainty: Adaptive Information Presentation for Statistical Dialogue Systems (2014, methodology): “The reward/evaluation function is constructed from the WoZ data, using a stepwise linear regression, following the paradise framework [61].”
Survey on evaluation methods for dialogue systems (2019, background): “In a follow-up study (Walker et al., 2000), the authors further investigated PARADISE’s ability to generalize to other systems and user populations and its predictive power.”
Neural Approaches to Conversational AI (2018, positive): “ning an appropriate reward function is not always obvious, and substantial domain knowledge is needed (Sec. 4.1). This inspires the use of 48 machine learning to ﬁnd a good reward function from data (Walker et al., 2000; Rieser and Lemon, 2008; Rieser et al., 2010; El Asri et al., 2012) which can be”