Affinity CNN: Learning Pixel-Centric Pairwise Relations for Figure/Ground Embedding

By Michael Maire, Takuya Narihira, Stella X. Yu. 2015

70 citations recorded by the source corpus.

Spectral embedding provides a framework for solving perceptual organization problems, including image segmentation and figure/ground organization. From an affinity matrix describing pairwise relationships between pixels, it clusters pixels into regions, and, using a complex-valued extension, orders pixels according to layer. We train a convolutional neural network (CNN) to directly predict the pair-wise relationships that define this affinity matrix. Spectral embedding then resolves these predictions into a globally-consistent segmentation and figure/ground organization of the scene. Experiments demonstrate significant benefit to this direct coupling compared to prior works which use explicit intermediate stages, such as edge detection, on the pathway from image to affinities. Our results suggest spectral embedding as a powerful alternative to the conditional random field (CRF)-based globalization schemes typically coupled to deep neural networks.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity (2022, background): “The pixel-level pairwise affinity can be either hand-constructed [7, 23, 24] or learned [16, 17, 28, 34, 35,40,50].”
Learning Affinity via Spatial Propagation Networks (2017, methodology): “[22] trained a deep CNN to directly predict the entities of an affinity matrix, which demonstrated good performance on image segmentation.”
Recurrent Pixel Embedding for Instance Grouping (2017, methodology): “, [62]), we use a pairwise loss but learn the embedding directly.”
SSAP: Single-Shot Instance Segmentation With Affinity Pyramid (2019, methodology): “The concept of learning pixel-pair affinity has been developed in many previous works [36, 23, 1, 4, 42] to facilitate semantic segmentations during training or postprocessing.”
Adaptive Affinity Field for Semantic Segmentation (2018, background): “Besides, low-level cues, such as affinity [25,26] and contour [27,28] have also been leveraged to encode image structures.”
Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network (2018, methodology): “[40] trained a deep CNN to directly predict the entities of an affinity matrix, which demonstrated good performance on image segmentation.”
Learning to group discrete graphical patterns (2017, unknown): “…employed deep learning architectures to group visual patterns appearing in natural images , in the context of image clustering [Xie et al. 2016; Yang et al. 2016], image classification [Greff et al. 2016], figure-ground segmentation [Maire et al. 2016], and contour detection [Maninis et al. 2016].”