Neural Prototype Trees for Interpretable Fine-grained Image Recognition

By Meike Nauta, Ron van Bree, Christin Seifert. 2020

369 citations recorded by the source corpus.

Prototype-based methods use interpretable representations to address the black-box nature of deep learning models, in contrast to post-hoc explanation methods that only approximate such models. We propose the Neural Prototype Tree (ProtoTree), an intrinsically interpretable deep learning method for fine-grained image recognition. ProtoTree combines prototype learning with decision trees, and thus results in a globally interpretable model by design. Additionally, ProtoTree can locally explain a single prediction by outlining a decision path through the tree. Each node in our binary tree contains a trainable prototypical part. The presence or absence of this learned prototype in an image determines the routing through a node. Decision making is therefore similar to human reasoning: Does the bird have a red throat? And an elongated beak? Then it’s a hummingbird! We tune the accuracy-interpretability trade-off using ensemble methods, pruning and binarizing. We apply pruning without sacrificing accuracy, resulting in a small tree with only 8 learned prototypes along a path to classify a bird from 200 species. An ensemble of 5 ProtoTrees achieves competitive accuracy on the CUB-200-2011 and Stanford Cars data sets. Code is available at github.com/M-Nauta/ProtoTree.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “[216] proposed a method for learning neural prototype trees based on a proto-”
This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition (2020, methodology): “Furthermore, because of the stand-alone nature of our method, it can be applied to any prototypical image recognition method, including ProtoPNet [11] and ProtoTree [29].”
Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions (2023, neutral): “…learn features that describe ‘prototypical concepts’ or ‘prototypes’ present in individual inputs to the model, including ProtoPNet [147], ProtoTree [148], ProtoPShare [149], Concept Bottleneck Models [150], Concept Activation Vectors [146], Concept Embedding Models [151] or Concept Atlases [152].”
From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI (2022, background): “Prototypes (Parts of) Representative examples, including concepts [127], influential training instances [90], prototypical parts [36, 179], nearest neighbors and criticisms [125].”