TCAV: R ELATIVE CONCEPT IMPORTANCE TESTING WITH L INEAR C ONCEPT A CTIVATION V ECTORS

2017

Subfields: Safety, Robustness & Interpretability

Citations in the corpus

• Scalable agent alignment via reward modeling: a research direction (2018, background): “…attend to when they make decisions (Greydanus et al., 2018), evaluating models’ sensitivity to the presence/intensity of high-level human concepts (Kim et al., 2017), optimizing a model to be more interpretable with humans in the loop (Lage et al., 2018), translating neural activations into…”
• Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, background): “For example, Concept Activation Vectors [54] are evaluated by training several versions of the same network on datasets that are synthesized to contain two different types of signals that can be used to determine the class (the image itself, and an overlaid piece of text which gives the class name…”
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, background): “The simplicity of the heuristic suggests that beyond immediate benefits for human trust in deep learning, techniques that enable interpretability—like the DkNN—will make powerful debugging tools for deep learning practitioners to better identify the limitations of their algorithms and models in a se”
• Concept whitening for interpretable image recognition (2020, methodology): “For comparison with posthoc methods like TCAV and IBD, we measure the output of their BN modules in the pretrained model, because the output of these layers are mean centered and normalized, which, as we discussed, are important properties for concept vectors.”
• Deep Neural Decision Trees (2018, background): “Some are model-agnostic (Ribeiro et al., 2016), while most are associated with a certain type of model, e.g., rule-based classifiers (Dash et al., 2015; Malioutov et al., 2017), nearest neighbour models (Kim et al., 2016), and neural networks (Kim et al., 2017).”
• Towards Robust Interpretability with Self-Explaining Neural Networks (2018, background): “Interpretable concept learning is a challenging task in its own right [Kim et al., 2017], and as other aspects of interpretability, remains ill-defined.”
• Regression Concept Vectors for Bidirectional Explanations in Histopathology (2018, methodology): “The explanations are evaluated on the basis of their statistical significance as proposed in [5].”
• Explaining Explanations to Society (2019, background): “The second class are systems that explain representation for DNNs either in terms of layers [26, 34], neurons [2] or vectors [14].”
• How Important Is a Neuron? (2018, background): “Finally, [16] takes a human-defined concept as input, finds hidden neurons that are predictive of this concept, and identifies the influence of these neurons on a certain prediction.”
• MetaPoison: Practical General-purpose Clean-label Data Poisoning (2020, background): “Thus, systems should rely on additional auxiliary measures, such as interpretability techniques (Kim et al., 2017), to make security-critical decisions.”
• Overlearning Reveals Sensitive Attributes (2019, background): “Other than an anecdotal mention in the acknowledgments paragraph of (Kim et al., 2017) that logit-layer activations leak non-label concepts, this phenomenon has never been described in the research literature.”