Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations
By Wolfgang Stammer, Patrick Schramowski, Kristian Kersting. 2020
139 citations recorded by the source corpus.
Most explanation methods in deep learning map importance estimates for a model’s prediction back to the original input space. These "visual" explanations are often insufficient, as the model’s actual concept remains elusive. Moreover, without insights into the model’s semantic concept, it is difficult —if not impossible— to intervene on the model’s behavior via its explanations, called Explanatory Interactive Learning. Consequently, we propose to intervene on a Neuro-Symbolic scene representation, which allows one to revise the model on the semantic level, e.g. "never focus on the color to make your decision". We compiled a novel confounded visual scene data set, the CLEVR-Hans data set, capturing complex compositions of different objects. The results of our experiments on CLEVR-Hans demonstrate that our semantic explanations, i.e. compositional explanations at a per-object level, can identify confounders that are not identifiable using "visual" explanations only. More importantly, feedback on this semantic level makes it possible to revise the model from focusing on these factors.
Subfields: Safety, Robustness & Interpretability, Knowledge Graphs & Representation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Safety, Robustness & Interpretability
- Nicholas A. Roy
- James Y. Zou
- Felix Hill
- James L. McClelland
- Maggie Haitian Wang
- Mert Yuksekgonul
- Andrew Kyle Lampinen
- Adam Santoro
In Knowledge Graphs & Representation
Citations in the corpus
- Tell me why! Explanations support learning relational and causal structure (2021, background): “Stammer et al. (2021) extend this approach in neurosymbolic models to intervene on symbolically-conveyed semantics rather than purely visual features.”
- Tell me why! - Explanations support learning of relational and causal structure (2021, background): “Stammer et al. (2021) extend this approach in neurosymbolic models to intervene on symbolically-conveyed semantics rather than purely visual features.”
- Post-hoc Concept Bottleneck Models (2022, background): “Right for the Right Concepts (Stammer et al., 2021) similarly fine-tune a neuro-symbolic model with user-provided feedback maps.”