Disentangled Representation Learning GAN for Pose-Invariant Face Recognition

By Luan Tran, Xi Yin, Xiaoming Liu. 2017

998 citations recorded by the source corpus.

The large pose discrepancy between two face images is one of the key challenges in face recognition. Conventional approaches for pose-invariant face recognition either perform face frontalization on, or learn a pose-invariant representation from, a non-frontal face image. We argue that it is more desirable to perform both tasks jointly to allow them to leverage each other. To this end, this paper proposes Disentangled Representation learning-Generative Adversarial Network (DR-GAN) with three distinct novelties. First, the encoder-decoder structure of the generator allows DR-GAN to learn a generative and discriminative representation, in addition to image synthesis. Second, this representation is explicitly disentangled from other face variations such as pose, through the pose code provided to the decoder and pose estimation in the discriminator. Third, DR-GAN can take one or multiple images as the input, and generate one unified representation along with an arbitrary number of synthetic images. Quantitative and qualitative evaluation on both controlled and in-the-wild databases demonstrate the superiority of DR-GAN over the state of the art.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Conditional variance penalties and domain shift robustness (2017, background): “In the computer vision literature, various works have used identity information to achieve pose invariance in the context of face recognition (Bartlett and Sejnowski, 1996; Tran et al., 2017).”
RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, background): “[131] proposed a Disentangled Representation learning Generative Adversarial Networks (DR-GAN) for pose-invariant face recognition.”
Interpreting the Latent Space of GANs for Semantic Face Editing (2019, methodology): “To achieve this goal, current methods required carefully designed loss functions [26, 10, 32], introduction of additional attribute labels or features [23, 36, 3, 34, 31], or special architectures [11, 30] to train new models.”
GANSpace: Discovering Interpretable GAN Controls (2020, background): “One approach to adding continuous control is to augment the GAN model itself with image labels at training time [14,21], which can improve generalization to new combinations [20].”
Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis (2017, methodology): “At this bottleneck layer, as in [33], we concatenate a 100-dim Gaussian random noise to v id to model variations other than pose and identity.”
Learning to Manipulate Individual Objects in an Image (2020, background): “Also, domain-specific knowledge could be utilized to learn disentangled representations [33, 32, 21].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “1 Face: Pose related: Disentangled representation learning GAN (DR-GAN) [324] is proposed for pose-invariant face recognition.”
Deep multi-view learning methods: A review (2021, methodology): “And the widely-used GAN has shown impressive results due to its characteristic of of adversarial training [36, 33, 68, 67].”
Self-supervised learning of a facial attribute embedding from video (2018, background): “…known (labelled) attribute information, e.g. for pose or expression, the embedding can be learned by training in a supervised manner to directly predict the attribute [28, 32, 46], or to generate images (of faces, cars, or other classes) at a new, known pose, expression, etc. [13, 25, 53, 59, 68].”
Face De-Spoofing: Anti-Spoofing via Noise Modeling (2018, methodology): “In this work, we analyze the properties of spoof noise and design a GAN-fashion generative model [32] to estimate the spoof noise pattern and peel it off the spoof image.”
Applications of Generative Adversarial Networks (GANs): An Updated Review (2019, background): “While much progress has been made to alleviate some of the challenges Change facial attributes DR-GAN [108] Change facial attributes SD-GAN [25] Change facial attributes SL-GAN [130] Change facial attributes VAE-GAN [59] Change facial attributes CAAE [140] Change facial attributes Age-cGAN [3]…”
InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs (2020, methodology): “To achieve this goal, existing methods require carefully designed loss functions [33], [34], [35], introduction of additional attribute labels [7], [36], [37], [38], [39], or special architectures [40], [41] to train new models.”
Towards Large-Pose Face Frontalization in the Wild (2017, methodology): “A recent method, DR-GAN [30], has been proposed concurrent to this work.”
Factorizing Knowledge in Neural Networks (2022, background): “The disentanglement are usually done through adversarial learning [58, 34, 40, 10] or variational auto-encoder [19, 7, 26].”
Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation (2021, methodology): “Instead of post-processing, studies have also been given to achieve disentanglement by incorporating extra regularization [7, 10, 30, 32, 34, 40, 48] in GAN training.”
A Survey on Generative Adversarial Networks: Variants, Applications, and Training (2020, methodology): “ge-to-image translation: Pix2pix [34], PAN [112], CycleGAN [113], Disco-GAN [114], Dual-GAN [115], StarGAN [116], UNIT [117], MUNIT [118], DRIT [119], DRIT++ [120] Face frontal view generation: DRGAN [121], TPGAN [122], FFGAN [123], FTGAN [125] Audio Speech and audio synthesis: Rank -GAN [150], VAW ”
A Unified Feature Disentangler for Multi-Domain Image Translation and Manipulation (2018, background): “Previous works [25, 15] had discovered that even the disentangled feature to the generator/decoder is binary during training, it can be considered as continuous variable during testing.”
FSA-Net: Learning Fine-Grained Structure Aggregation for Head Pose Estimation From a Single Image (2019, background): “It could also be used to improve or provide extra information for other problems such as identity recognition [39], expression recognition [46], or attention detection [8].”
Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning (2020, methodology): “Bottom: Face frontalization results compared with PIM [52], TP-GAN [21], DRGAN [47], Hassner et al.”