A Review of User Interface Design for Interactive Machine Learning

By John J. Dudley, Per Ola Kristensson. ACM Trans. Interact. Intell. Syst., 2018

377 citations recorded by the source corpus.

Interactive Machine Learning (IML) seeks to complement human perception and intelligence by tightly integrating these strengths with the computational power and speed of computers. The interactive process is designed to involve input from the user but does not require the background knowledge or experience that might be necessary to work with more traditional machine learning techniques. Under the IML process, non-experts can apply their domain knowledge and insight over otherwise unwieldy datasets to find patterns of interest or develop complex data-driven applications. This process is co-adaptive in nature and relies on careful management of the interaction between human and machine. User interface design is fundamental to the success of this approach, yet there is a lack of consolidated principles on how such an interface should be implemented. This article presents a detailed review and characterisation of Interactive Machine Learning from an interactive systems perspective. We propose and describe a structural and behavioural model of a generalised IML system and identify solution principles for building effective interfaces for IML. Where possible, these emergent solution principles are contextualised by reference to the broader human-computer interaction literature. Finally, we identify strands of user interface research key to unlocking more efficient and productive non-expert interactive machine learning applications.

Subfields: Computer Science Applications, Computer Vision and Pattern Recognition, Management Information Systems

View Paper: https://www.semanticscholar.org/paper/ad906fb0dc38f0e7a59f48c968647bd81ed778bb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Computer Vision and Pattern Recognition

In Management Information Systems

Citations in the corpus

Interactive Model Cards: A Human-Centered Approach to Model Documentation (2022, background): “Prior work has shown that non-expert analysts benefit from interacting with machine learning models and their data [1, 19, 59].”
Bridging the Gap Between Ethics and Practice (2020, background): “These guidelines build on a long history of user interface design [104, 100] and newer research on designing interfaces for HCAI systems [29, 4, 103].”
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy (2020, neutral): “Other guidelines (Horvitz, 1999; Dudley & Kristensson, 2018) paved the way for Microsoft’s Guidelines for AI-Human Interaction (https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction/), which has 18 guidelines for initial use, normal use, coping with problems, and…”
Informed Machine Learning – A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems (2019, background): “In order to obtain rich feedback, careful design of visual analytics guided machine learning is essential [145], [162], [163] leading to new principles for the interface design of interactive learning systems [164].”
Semi-Automated Data Labeling (2020, background): “By ensuring that the labeler is regularly updated on the performance of the model, the checkpoints provide ‘temporal speciﬁcity’ and manage user expectations throughout the interaction, a guiding design principle for interactive machine learning Dudley and Kristensson (2018).”
A Survey of Human-in-the-loop for Machine Learning (2021, background): “• For a system with human-computer interaction, users’ expectations of experience usually take precedence over performance [157, 158].”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, background): “Previously, various models and guidelines for the design and evaluation of AI-infused interactive user interfaces [7, 54] and visual analytics systems [155] have been proposed to help designers through the design process.”
Human-in-the-loop machine learning: a state of the art (2022, background): “Dudley and Kristensson (2018) also emphasize the iterative part in their definition of IML as “an interaction paradigm in which a user or user group iteratively builds and refines a mathematical model to describe a concept through iterative cycles of input and review”.”
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts (2023, methodology): “How can BotDesigner enable users to inspect a prompt’s impact on a full set of conversations, while each of these conversations unfolds diferently depending on the user’s utterances and the LLM’s probabilistic outputs? To address the challenge in tracking prompt’s efects, BotDesigner builds on tool ”
Toward User-Driven Sound Recognizer Personalization with People Who Are d/Deaf or Hard of Hearing (2021, methodology): “, adjusting parameters) [19], and future work intending to leverage the benefts of human-in-the-loop systems for DHH users should explore alternatives.”