Unsupervised Joint Alignment of Complex Images

By Gary B. Huang, Vidit Jain, Eric Learned-Miller. IEEE International Conference on Computer Vision, 2007

383 citations recorded by the source corpus.

Many recognition algorithms depend on careful positioning of an object into a canonical pose, so the position of features relative to a fixed coordinate system can be examined. Currently, this positioning is done either manually or by training a class-specialized learning algorithm with samples of the class that have been hand-labeled with parts or poses. In this paper, we describe a novel method to achieve this positioning using poorly aligned examples of a class with no additional labeling. Given a set of unaligned examplars of a class, such as faces, we automatically build an alignment mechanism, without any additional labeling of parts or poses in the data set. Using this alignment mechanism, new members of the class, such as faces resulting from a face detector, can be precisely aligned for the recognition process. Our alignment method improves performance on a face recognition task, both over unaligned images and over images aligned with a face alignment algorithm specifically developed for and trained on hand-labeled face images. We also demonstrate its use on an entirely different class of objects (cars), again without providing any information about parts or pose to the learning algorithm.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b89a0ef7f42da284ca8e403812ad871297992716

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Weakly Supervised Manifold Learning for Dense Semantic Object Correspondence (2017, methodology): “Recently, joint image-set alignment has been used to tackle correspondence problem using supervised and fully unsupervised methods as shown by Learned-Miller’s congealing procedure[13].”
RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images (2010, background): “Learned-Miller’s influential congealing algorithm seeks an alignment that minimizes the sum of entropies of pixel values at each pixel in the batch of aligned images [16, 14].”
Labeled Faces in the Wild: A Survey (2016, methodology): “3 Unsupervised joint alignment of complex images, 2007 [45]”
Learning from ambiguously labeled images (2009, methodology): “We experiment with a subset of the publicly available Labeled Faces in the Wild [10] dataset.”
Age and Gender Estimation of Unfiltered Faces (2014, methodology): “In [55], the “funneling” approach to image alignment was proposed, based on sets of unaligned face image examples, and its impact on recognition performance was demonstrated.”
Attribute and simile classifiers for face verification (2009, methodology): “Not surprisingly, LFW has proven difficult for automatic face verification methods [25, 34, 17, 18, 19].”
Effective Unconstrained Face Recognition by Combining Multiple Descriptors and Learned Background Statistics (2011, neutral): “Since its recent publication, a lot of attention has been focused on improving performance on the benchmarks associated with the LFW database (see, e.g., [1], [2], [3], [6], [14], [15], [16], [17]).”
Unsupervised Object Segmentation by Redrawing (2019, methodology): “A subpart of the funneled version [24] has been segmented and manually annotated [27], providing 2927 groundtruth masks.”
Describable Visual Attributes for Face Verification and Image Search (2011, neutral): “Various 2D alignment strategies have been applied to LFW—aligning all faces [26] to each other or aligning each pair of images to be considered for verification [19], [38], [25].”
Probabilistic Elastic Matching for Pose Variant Face Verification (2013, methodology): “, raw face images extracted from the Viola-Jones face detector [24], indeed outperforms the state-of-the-art algorithm, such as the bioinspired V1 features with multiple kernel learning applied to faces aligned with the funneling method [15] under the most restricted protocol in LFW.”
Movie/Script: Alignment and Parsing of Video and Text Transcription (2008, background): “Several recent papers [1, 2] have successfully collected very large-scale, diverse datasets of faces “in the wild” using weakly supervised techniques.”
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments (2008, background): “Shortly after the publication of the original paper describ ing Faces in the Wild, a variety of authors started to inquire about using such a distribution of face images in their work [14], [15], [25], [28].”
Deep Learning for Semantic Part Segmentation with High-Level Guidance (2015, methodology): “Recently Deep Convolutional Neural Networks (DCNNs) have delivered excellent results in a broad range of computer vision problems, including but not limited to image classification Krizhevsky et al. (2012); Sermanet et al.”
GAN-Supervised Dense Visual Alignment (2021, background): “, [75,92]), with the seminal work of Congealing [31, 47] establishing unsupervised joint alignment as a research problem.”
Do Convnets Learn Correspondence? (2014, methodology): “Congealing [6] is an unsupervised joint alignment method based on an entropy objective.”
FlowWeb: Joint image set alignment by weaving consistent, pixel-wise correspondences (2015, background): “Congealing uses sequential optimization to gradually lower the entropy of the intensity distribution of the entire image set by continuously warping each image via a parametric transformation (e.g. afﬁne).”