Deep Feature Factorization For Concept Discovery

By Edo Collins, Radhakrishna Achanta, Sabine Süsstrunk. 2018

111 citations recorded by the source corpus.

We propose Deep Feature Factorization (DFF), a method capable of localizing similar semantic concepts within an image or a set of images. We use DFF to gain insight into a deep convolutional neural network's learned features, where we detect hierarchical cluster structures in feature space. This is visualized as heat maps, which highlight semantically matching regions across a set of images, revealing what the network `perceives' as similar. DFF can also be used to perform co-segmentation and co-localization, and we report state-of-the-art results on these tasks.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

GAN Inversion: A Survey (2021, methodology): “[188] for concept discovery and produces state-of-the-art results on co-segmentation and co-localization tasks.”
Editing in Style: Uncovering the Local Semantics of GANs (2020, methodology): “Deep feature factorization (DFF) [6] is a recent method that explains a convolutional neural network’s (CNN) learned representation through a set of saliency maps, extracted by factorizing a matrix of hidden layer activations.”
Deep Feature Factorization for Content-Based Image Retrieval and Localization (2019, methodology): “In this paper, we show that deep feature factorization (DFF) [1] is effective for both deriving image descriptors for retrieval, and for subsequent localization of the relevant region within each of the top ranked retrieved images.”
Detecting Memorization in ReLU Networks (2018, methodology): “In the context of deep convolutional neural networks, Collins et al. (2018) applied NMF to the activations of an image classifier and showed that the result gives a decomposition into semantic parts, which benefits from the transformation invariance learned by the neural network.”
Unsupervised Semantic Segmentation by Distilling Feature Correspondences (2022, background): “Though most works aim to generate a single vector for an image, many works show that intermediate dense features are semantically relevant (Hamilton et al., 2021; Collins et al., 2018; Zhou et al., 2016).”