Teaching Compositionality to CNNs

By Austin Stone, Hua-Yan Wang, Michael Stark, D. Scott Phoenix, Dileep George. 2017

58 citations recorded by the source corpus.

Convolutional neural networks (CNNs) have shown great success in computer vision, approaching human-level performance when trained for specific tasks via application-specific loss functions. In this paper, we propose a method for augmenting and training CNNs so that their learned features are compositional. It encourages networks to form representations that disentangle objects from their surroundings and from each other, thereby promoting better generalization. Our method is agnostic to the specific details of the underlying CNN to which it is applied and can in principle be used with any CNN. As we show in our experiments, the learned representations lead to feature activations that are more localized and improve performance over non-compositional baselines in object recognition tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Learning of Neural Networks to Explain Neural Networks (2018, background): “A new trend related to network interpretability is to learn networks with disentangled, interpretable representations [12, 31, 21].”
Interpretable Convolutional Neural Networks (2017, background): “[27] learned CNN representations with better object compositionality, and Liao et al.”
Combining Compositional Models and Deep Networks For Robust Object Classification under Occlusion (2019, background): “Related approaches propose to regularize the convolution ﬁlters to be sparse [18], or to enforce the activations in the feature maps to be disentangled for different objects [17].”
AGQA: A Benchmark for Compositional Spatio-Temporal Reasoning (2021, background): “Compositional reasoning is fundamental to understanding visual events [30, 37] and has been sought after recently by a number of papers [47, 55, 57, 13, 24].”
Zero-Shot Learning from scratch (ZFS): leveraging local compositional representations (2020, methodology): “As a ﬁrst good step to solving ZFS, we follow intuitions from works on compositionality and locality (Tokmakov et al., 2018; Stone et al., 2017), focusing on methods that encourage good local representations.”
Locality and compositionality in zero-shot learning (2019, background): “Speciﬁcally, we explore the role of compositionality and locality (Tokmakov et al., 2018; Stone et al., 2017) as two principles that lead to good generalization.”
Learning Compositional Representations for Few-Shot Recognition (2018, background): “In [37] the authors address the notion of spatial compositionality, proposing to constrain network representations of objects in an image to be independent from each other and from the background.”
Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition Under Occlusion (2020, background): “Other related works propose to regularize the convolution kernels to be sparse [52], or to force feature activations to be disentangled for different objects [51].”
Compositional Convolutional Neural Networks: A Deep Architecture With Innate Robustness to Partial Occlusion (2020, background): “Related works proposed to regularize the convolution kernels to be sparse [20], or to force feature activations to be disentangled for different objects [19].”