Learning invariance through imitation

By Graham W. Taylor, Ian Spiro, C. Bregler, Rob Fergus. 2011

54 citations recorded by the source corpus.

Supervised methods for learning an embedding aim to map high-dimensional images to a space in which perceptually similar observations have high measurable similarity. Most approaches rely on binary similarity, typically defined by class membership where labels are expensive to obtain and/or difficult to define. In this paper we propose crowd-sourcing similar images by soliciting human imitations. We exploit temporal coherence in video to generate additional pairwise graded similarities between the user-contributed imitations. We introduce two methods for learning nonlinear, invariant mappings that exploit graded similarities. We learn a model that is highly effective at matching people in similar pose. It exhibits remarkable invariance to identity, clothing, background, lighting, shift and scale.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “UL can also serve to extract invariant features from different data items (e.g., Becker, 1991) through coupled NNs observing two different inputs (Schmidhuber & Prelinger, 1992), also called Siamese NNs (e.g., Bromley et al., 1993; Chen & Salman, 2011; Hadsell, Chopra, & LeCun, 2006; Taylor, Spiro, ”
Learning Fine-Grained Image Similarity with Deep Ranking (2014, methodology): “For example, in [12, 22], two images are considered similar as long as they belong to the same category.”
Graph-RISE: Graph-Regularized Image Semantic Embedding (2019, background): “Several previous works consider category-level image semantics [9, 26], in which two images are considered semantically similar if they belong to the same category.”
Learning visual feature descriptors for dynamic lighting conditions (2014, methodology): “However, with auto-encoders the goal is to produce a lower-dimensional embedding that can be decoded with minimal reconstruction error, which does not necessarily produce embeddings where Euclidean distance is useful for discrimination [28].”
Watching Unlabeled Video Helps Learn New Human Actions from Very Few Labeled Snapshots (2013, methodology): “In parallel, many methods to estimate body pose have been developed, including techniques using nonlinear manifolds to represent the complex space of joint configurations [12, 32, 3, 16, 28, 29]; in contrast to our work, such methods assume silhouette (backgroundsubtracted) inputs and/or derive mode”
Object Detection With Deep Learning: A Review (2018, background): “Among the pre-related works [44], [75], [76], the most representative one is Overfeat [44].”
Contrastive Representation Learning: A Framework and Review (2020, methodology): “While there have been approaches to using probabilistic approaches to learning metric embeddings [98], most successful applications up to now all use the energy-based pair or triplet loss due to the computational requirements to compute the normalisation constant in probabilistic loss.”
On Self-Contact and Human Pose (2021, background): “[42] crowd-source images of people in the same pose by imitation.”
Learning by coincidence: Siamese networks and common variable learning (2018, methodology): “AC CE PT ED M AN US CR IP T network structures such as convolutional structures [26], as discussed in works 390 such as [10] and [6]; in such cases, the networks can also be represented more efficiently (e.”
Ultra Fine-Grained Image Semantic Embedding (2020, background): “Several previous works consider category-level image semantics [9, 26], in which two images are considered semantically similar if they belong to the same category.”