Compositional Convolutional Neural Networks: A Deep Architecture With Innate Robustness to Partial Occlusion

By Adam Kortylewski, Ju He, Qing Liu, Alan L. Yuille. 2020

110 citations recorded by the source corpus.

Recent work has shown that deep convolutional neural networks (DCNNs) do not generalize well under partial occlusion. Inspired by the success of compositional models at classifying partially occluded objects, we propose to integrate compositional models and DCNNs into a unified deep model with innate robustness to partial occlusion. We term this architecture Compositional Convolutional Neural Network. In particular, we propose to replace the fully connected classification head of a DCNN with a differentiable compositional model. The generative nature of the compositional model enables it to localize occluders and subsequently focus on the non-occluded parts of the object. We conduct classification experiments on artificially occluded images as well as real images of partially occluded objects from the MS-COCO dataset. The results show that DCNNs do not classify occluded objects robustly, even when trained with data that is strongly augmented with partial occlusions. Our proposed model outperforms standard DCNNs by a large margin at classifying partially occluded objects, even when it has not been exposed to occluded objects during training. Additional experiments demonstrate that CompositionalNets can also localize the occluders accurately, despite being trained with class labels only. The code and data used in this work are publicly available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Alan L. Yuille
2. Philip H. S. Torr
3. Xinggang Wang
4. Song Bai
5. Xiang Bai
6. Serge J. Belongie
7. Bernt Schiele
8. Trevor Darrell

In Computer Vision and Pattern Recognition

1. Xinggang Wang
2. Xiang Bai
3. Alan L. Yuille
4. Luc Van Gool
5. Chunhua Shen
6. Philip H. S. Torr
7. Ming-Hsuan Yang
8. Dacheng Tao

In Model Architectures

1. Alan L. Yuille
2. Philip H. S. Torr
3. Song Bai
4. Xiang Bai
5. Xiaoyu Liu
6. Qing Liu
7. Yan Gao
8. Yihong Sun

In Safety, Robustness & Interpretability

1. Song Bai
2. Philip H. S. Torr
3. Serge J. Belongie
4. Alan L. Yuille
5. Adam Kortylewski

Citations in the corpus

• Occluded Video Instance Segmentation: A Benchmark (2021, methodology): “Different from the full-DNN paradigm described above, Some methods[40,41,42] integrate compositional models and deep convolutional neural networks into a unified model which is more robust to partial occlusions.”
• Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition Under Occlusion (2020, background): “ents that demonstrate the ability of CompositionalNets to localize occluders accurately, despite being trained with class labels only. Finally, we note that this article extends the conference papers [27,55] in multiple ways: (1) We present the models of both papers coherently in a common theoretica”