Teaching Categories to Human Learners with Visual Explanations

By Oisin Mac Aodha, Shihan Su, Yuxin Chen, Pietro Perona, Yisong Yue. 2018

74 citations recorded by the source corpus.

We study the problem of computer-assisted teaching with explanations. Conventional approaches for machine teaching typically only provide feedback at the instance level e.g., the category or label of the instance. However, it is intuitive that clear explanations from a knowledgeable teacher can significantly improve a student's ability to learn a new concept. To address these existing limitations, we propose a teaching framework that provides interpretable explanations as feedback and models how the learner incorporates this additional information. In the case of images, we show that we can automatically generate explanations that highlight the parts of the image that are responsible for the class label. Experiments on human learners illustrate that, on average, participants achieve better test set performance on challenging categorization tasks when taught with our interpretable approach compared to existing methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning to Explain With Complemental Examples (2018, background): “Prototype selection [25, 7, 13, 16, 25, 10] or machine teaching [18] can be considered as example-based explanations.”
Counterfactual Visual Explanations (2019, background): “To our knowledge, only one other work (Mac Aodha et al., 2018) uses visual explanations for machine teaching.”
Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning (2018, background): “A successful FGVC model should be able to discriminate categories with subtle differences, which presents formidable challenges for the model design yet also provides insights to a wide range of applications such as rich image captioning [3], image generation [5], and machine teaching [27, 37].”
On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection (2018, background): “One possible direction is to develop automatic tutorials to teach the intuitions behind important features, which is related to machine teaching [51, 68, 82].”
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2020, background): “AI explanations are useful in many scenarios: communicating model predictions [17, 30, 33, 56], teaching humans tasks like translation [48] or content moderation [28], augmenting human analysis procedure [28] or creativity [14], legal imperatives [51, 64], and others.”