Disentangling Nonlinear Perceptual Embeddings With Multi-Query Triplet Networks

By Andreas Veit, Serge J. Belongie, Theofanis Karaletsos. 2016

8 citations recorded by the source corpus.

In typical perceptual tasks, higher-order concepts are inferred from visual features to assist with perceptual decision making. However, there is a multitude of visual concepts which can be inferred from a single stimulus. When learning nonlinear embeddings with siamese or triplet networks from similarities, we typically assume they are sourced from a single visual concept. In this paper, we are concerned with the hypothesis that it can be potentially harmful to ignore the heterogeneity of concepts affiliated with observed similarities when learning these embedding networks. We demonstrate empirically that this hypothesis holds and suggest an approach that deals with these shortcomings, by combining multiple notions of similarities in one compact system. We propose Multi-Query Networks (MQNs) that leverage recent advances in representation learning on factorized triplet embeddings in combination with Convolutional Networks in order to learn embeddings differentiated into semantically distinct subspaces, which are learned with a latent space attention mechanism. We show that the resulting model learns visually relevant semantic subspaces with features that do not only outperform single triplet networks, but even sets of concept specific networks.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Learning Deep Disentangled Embeddings with the F-Statistic Loss (2018, background): “, 2015) or are ordered by factor-specific similarity (Karaletsos et al., 2015; Veit et al., 2016).”
Structured Disentangled Representations (2018, background): “…p ( z ) , which in turn ensures realistic samples x ∼ p θ ( x ) from the generative model. variables [Kingma et al., 2014, Siddharth et al., 2017], triplet supervision [Karaletsos et al., 2015, Veit et al., 2016], or batch-level factor invariances [Kulkarni et al., 2015, Boucha-court et al., 2017].”
A Survey of Inductive Biases for Factorial Representation-Learning (2016, background): “A very similar approach is taken in [56], which demonstrated the model’s ability to learn factorial representations on other datasets, such as stylized fonts and images of shoes.”
Automatic Spatially-Aware Fashion Concept Discovery (2017, methodology): “[29] also learns concept subspaces based on an attention mechanism, they heavily rely on richly annotated data, while our method is fully automatic and annotation free.”