Interactive optimization for steering machine classification

By Ashish Kapoor, Bongshin Lee, Desney S. Tan, Eric Horvitz. 2010

166 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Information Systems and Management, Optimization & Training, Human-Computer Interaction, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Information Systems and Management

In Optimization & Training

In Human-Computer Interaction

In Signal Processing

Citations in the corpus

Rewriting a Deep Generative Model (2020, background): “Interactive machine learning systems aim to improve training through human interaction in labeling [15,22,68], or by allowing a user to to aid in the model optimization process via interactive feature selection [19,27,64,50] or model and hyperparameter selection [39,61,37].”
Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks (2016, background): “In [24], the authors present a visualization system for feed-forward neural networks with the goal of interpretation, and in [13], the authors give a user-interface for tuning the learning itself.”
Interactive and interpretable machine learning models for human machine collaboration (2015, background): “Even if the interaction is simplified to the level of performance metrics (confusion matrix) [77], it still requires the user to understand the confusion matrix.”
Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models (2018, background): “Typical solutions include improving hyperparameters [9,13], features [6,26,43], or training instances [29] and investigating multiple models in order to acquire an optimal model ensemble [35,38,43].”
A Review of User Interface Design for Interactive Machine Learning (2018, methodology): “ManiMatrix (Kapoor et al. 2010) allows users to interact directly with the classifier’s confusion matrix and thereby steer classification behaviour.”
The Role of Explanations in Assessing and Correcting Personalized Intelligent Agents (2012, background): “Recent research has made inroads into supporting this type of functionality [1, 2, 5, 7], but debugging can be difficult for even trained software developers—helping end users, who have knowledge of neither software engineering nor machine learning, is no trivial task.”
Understanding Hidden Memories of Recurrent Neural Networks (2017, methodology): “These methods include: visual classiﬁcation [6,32,48], visual optimization [24,45], and visual feature engineering [4,41,49].”
Principles of Explanatory Debugging to Personalize Interactive Machine Learning (2015, background): “This principle builds upon research showing that end users who received explanations about a learning system’s reasoning were able to better personalize classifiers and recommenders [4, 18, 24].”
Do Convolutional Neural Networks Learn Class Hierarchy? (2017, methodology): “Confusion matrices have been utilized to manipulate decision boundaries as in ManiMatrix [33], to combine multiple classiﬁers as in EnsembleMatrix [65], and to examine impact of model changes as in BaobabView [68].”
LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks (2016, background): “[15] give a user-interface for tuning the learning itself.”
Interacting with Predictions: Visual Inspection of Black-box Machine Learning Models (2016, background): “Some recent interactive machine learning tools [1, 2, 3, 4, 19, 21, 26, 29, 32]) are more algorithm agnostic, but depend on general performance measures like confusion matrices, area under ROC curve (AUC) measures, result distribution of data points, and feature weights according to model independen”
How the Experts Do It: Assessing and Explaining Agent Behaviors in Real-Time Strategy Games (2017, background): “[10] also showed that explaining AI increased user satisfaction and interacting with the explanations enabled users to construct classifiers that were more aligned with target preferences.”
Human-in-the-loop machine learning: a state of the art (2022, background): “Some of these authors (Kapoor et al. 2010) continued on this path with, ManiMa-trix , an interactive system that allowed interactive refinement of classification boundaries in a multiclass setting.”
Tell me more?: the effects of mental model soundness on personalizing an intelligent agent (2012, background): “Recent research has made inroads into supporting this type of functionality [1,11,14,16].”
Human model evaluation in interactive supervised learning (2011, background): “For example, ManiMatrix allows users to manipulate a confusion matrix to interactively steer a model’s performance to reflect their priorities [16], and EnsembleMatrix harnesses human users to optimize ensembles of learners [20].”
iBCM: Interactive Bayesian Case Model Empowering Humans via Intuitive Interaction (2015, background): “For example, in classification tasks, [19] users specified their preferred confusion matrix for classifications.”