Toward harnessing user feedback for machine learning

By Simone Stumpf, Vidya Rajaram, Lida Li, Margaret M. Burnett, Thomas G. Dietterich, Erin Sullivan, Russell Drummond, Jonathan L. Herlocker. 2007

180 citations recorded by the source corpus.

There has been little research into how end users might be able to communicate advice to machine learning systems. If this resource--the users themselves--could somehow work hand-in-hand with machine learning systems, the accuracy of learning systems could be improved and the users' understanding and trust of the system could improve as well. We conducted a think-aloud study to see how willing users were to provide feedback and to understand what kinds of feedback users could give. Users were shown explanations of machine learning predictions and asked to provide feedback to improve the predictions. We found that users had no difficulty providing generous amounts of feedback. The kinds of feedback ranged from suggestions for reweighting of features to proposals for new features, feature combinations, relational features, and wholesale changes to the learning algorithm. The results show that user feedback has the potential to significantly improve machine learning systems, but that learning algorithms need to be extended in several ways to be able to assimilate this feedback.

Subfields: Artificial Intelligence, Computer Science Applications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

Citations in the corpus

Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, methodology): “Authors Kulesza, Stumpf et al., 2011; Stumpf et al., 2007 AI System Bayesian classification algorithm for text classification.”
Anchors: High-Precision Model-Agnostic Explanations (2018, background): “Compared to other interpretable options, rules fare well; users prefer, trust and understand rules better than alternatives (Lim, Dey, and Avrahami 2009; Stumpf et al. 2007), in particular rules similar to anchors.”
Power to the People: The Role of Humans in Interactive Machine Learning (2014, methodology): “For example, Stumpf et al. (2007) conducted an experiment to understand the types of input end-users might provide to machine learning systems if unrestricted by the interface.”
FeatureInsight: Visual support for error-driven feature ideation in text classification (2015, result): “Stumpf et al. used a compare and contrast approach to help users give openended feedback to an email classification system [18].”
Interacting meaningfully with machine learning systems: Three experiments (2009, background): “An early presentation of Experiment #1 appeared in Proceedings of ACM IUI’07 ( Stumpf et al. 2007 ).”
An adaptive annotation approach for biomedical entity and relation recognition (2016, background): “On top of simple user feedback in [21], such as accepting and rejecting suggestions, complex feedback like choosing the best features, suggestions for the re-weighting of features, proposing new features and combining features remarkably improve the system.”
Questioning the AI: Informing Design Practices for Explainable AI User Experiences (2020, unknown): “Such a practice has already been engaged in interactive ML [6, 86] and "socially guided machine learning" [89].”
Explanatory Debugging: Supporting End-User Debugging of Machine-Learned Programs (2010, background): “As others [19] have reported, many participants had trouble understanding how absence of something played a role in the program’s decisionmaking.”
Toward establishing trust in adaptive agents (2008, background): “User studies focused solely on understanding machine learning [21, 25] have looked at how explanations can increase acceptance and usability of these learning algorithms in isolation, by testing user understanding of a variety of machine learning algorithms when explanations of various forms are ava”
Designing Contestability: Interaction Design, Machine Learning, and Mental Health (2017, background): “For example, [25] found strong user preference for, and trust in, models that exhibit “sound” (i.e., human comprehensible) reasoning and “clear communication” about decision making.”
The Role of Explanations in Assessing and Correcting Personalized Intelligent Agents (2012, background): “…of “more intelligent” intelligent agents (4). different types of explanations (keyword-based, rule-based, and similarity-based) that a machine learning systems could provide to end users regarding why it was behaving in a particular manner, as well as user reactions to these explanations [9].”
Tell me more?: the effects of mental model soundness on personalizing an intelligent agent (2012, background): “Prior work has focused on how an intelligent agent can explain itself to end users [9,13,15,22,27,28], and how end users might act upon such explanations to debug their intelligent agents [1,11,14,16,24].”
Explaining models: an empirical study of how explanations impact fairness judgment (2019, background): “Work on how people perceive explanations of ML systems is a growing area [4, 20, 25, 32] which aims to inform the choices and design of explanations for particular systems or tasks.”