Interacting meaningfully with machine learning systems: Three experiments

By Simone Stumpf, Vidya Rajaram, Lida Li, Weng-Keen Wong, Margaret M. Burnett, Thomas G. Dietterich, Erin Sullivan, Jonathan L. Herlocker. 2009

225 citations recorded by the source corpus.

Although machine learning is becoming commonly used in today's software, there has been little research into how end users might interact with machine learning systems, beyond communicating simple ''right/wrong'' judgments. If the users themselves could work hand-in-hand with machine learning systems, the users' understanding and trust of the system could improve and the accuracy of learning systems could be improved as well. We conducted three experiments to understand the potential for rich interactions between users and machine learning systems. The first experiment was a think-aloud study that investigated users' willingness to interact with machine learning reasoning, and what kinds of feedback users might give to machine learning systems. We then investigated the viability of introducing such feedback into machine learning systems, specifically, how to incorporate some of these types of user feedback into machine learning systems, and what their impact was on the accuracy of the system. Taken together, the results of our experiments show that supporting rich interactions between users and machine learning systems is feasible for both user and machine. This shows the potential of rich human-computer collaboration via on-the-spot interactions as a promising direction for machine learning systems and users to collaboratively share intelligence.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Manipulating and Measuring Model Interpretability (2018, background): “[97] Simone Stumpf, Vidya Rajaram, Lida Li, Weng-Keen Wong, Margaret Burnett, Thomas Dietterich, Erin Sullivan, and Jonathan Herlocker.”
FeatureInsight: Visual support for error-driven feature ideation in text classification (2015, methodology): “However, when training data is limited (e.g., for personalized tasks [13] or when domain expertise is required and prohibitively expensive to obtain [14]), automatically generated features can result in over-fitting [1]; in these scenarios, human involvement in feature engineering can improve…”
Visual Interaction with Deep Learning Models through Collaborative Semantic Inference (2019, background): “While some guidelines have been developed for interaction design for machine learning [2,25,75,86], they focus on designing machine learning methods where the model performs complementary tasks to the user, or where the user interacts with black-box models.”
Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models (2022, background): “Prior studies have examined how users interact with imperfect AI [11, 23, 26–28, 35, 37, 45].”
Toolkit to support intelligibility in context-aware applications (2010, methodology): “, [38]), we built different applications for each model type to demonstrate the generality of the Intelligibility Toolkit to provide explanations across application domains.”
Learning from a learning thermostat: lessons for intelligent systems for the home (2013, background): “been done into how to design interfaces that render system behavior more intelligible [1,12,24], such research has yet”
Interpreting Interpretability: Understanding Data Scientists' Use of Interpretability Tools for Machine Learning (2020, background): “[56] study the viability of including rich human feedback in an ML-based email filtering system, finding lack of common language between humans and machines to be a key challenge.”
Explanation-Based Human Debugging of NLP Models: A Survey (2021, methodology): “…papers, we started from some pivotal EBHD work, e.g., (Kulesza et al., 2015; Ribeiro et al., 2016; Teso and Kersting, 2019), and added EBHD papers citing or being cited by the pivotal work, e.g., (Stumpf et al., 2009; Kulesza et al., 2010; Lertvittayakumjorn et al., 2020; Yao et al., 2021).”
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2020, background): “, explanations can lead humans to either follow incorrect AI suggestions or ignore the correct ones [13, 69].”
Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models (2019, background): “Previous work demonstrates that interaction between users and machine learning systems is a promising direction for collaboratively sharing intelligence [60].”
Tell me more?: the effects of mental model soundness on personalizing an intelligent agent (2012, background): “There has been recent i terest in supporting the debugging of intelligent agents reasoning [1,11,13,14,16,25], but the mental models users build while attempting t his task have received little attention.”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, methodology): “Stumpf et al. [194] designed experiments to investigate meaningful explanations and interactions to hold users accountable by machine learning algorithms.”
Human-in-the-loop Debugging Deep Text Classifiers (2020, methodology): “To rectify such problems, there have been attempts to enable humans to fix the trained models (i.e., to perform model debugging) (Stumpf et al., 2009; Teso and Kersting, 2019).”
Evaluating XAI: A comparison of rule-based and example-based explanations (2021, background): “spam-filter over a rule-based [48], while they prefer spam-filter explanations to be rule-based [49].”
Will You Accept an Imperfect AI?: Exploring Designs for Adjusting End-user Expectations of AI Systems (2019, background): “A significant body of research exists on the topic of intel-ligibility of AI systems in particular [1, 35, 40, 46].”
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders (2019, background): “…there are some studies seeking to help non-expert end-users interpret and interact with ML models, they focus on applications such as image classification (e.g. [8]), translation (e.g. [24]), text mining and text classification (e.g. [ 33, 36, 47]), and context-aware systems (e.g. [5 , 34]).”
Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda (2018, background): “With the increasing use of machine learning, Intelligent User Interfaces began to focus on providing explanations to debug learned models [100, 101, 162].”