LaFIn: Generative Landmark Guided Face Inpainting

By Yang Yang, Xiaojie Guo, Jiayi Ma, Lin Ma, Haibin Ling. 2019

40 citations recorded by the source corpus.

It is challenging to inpaint face images in the wild, due to the large variation of appearance, such as different poses, expressions and occlusions. A good inpainting algorithm should guarantee the realism of output, including the topological structure among eyes, nose and mouth, as well as the attribute consistency on pose, gender, ethnicity, expression, etc. This paper studies an effective deep learning based strategy to deal with these issues, which comprises of a facial landmark predicting subnet and an image inpainting subnet. Concretely, given partial observation, the landmark predictor aims to provide the structural information (e.g. topological relationship and expression) of incomplete faces, while the inpaintor is to generate plausible appearance (e.g. gender and ethnicity) conditioned on the predicted landmarks. Experiments on the CelebA-HQ and CelebA datasets are conducted to reveal the efficacy of our design and, to demonstrate its superiority over state-of-the-art alternatives both qualitatively and quantitatively. In addition, we assume that high-quality completed faces together with their landmarks can be utilized as augmented data to further improve the performance of (any) landmark predictor, which is corroborated by experimental results on the 300W and WFLW datasets.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Zuxuan Wu
2. Yu-Gang Jiang
3. Eric I-Chao Chang
4. Yilun Sheng
5. Jingjing Chen
6. Yue Dong
7. Junke Wang
8. Xiao Liang

In Generative Models

1. Eric I-Chao Chang
2. Yu-Gang Jiang
3. Yilun Sheng
4. Jingjing Chen
5. Yue Dong
6. Xiao Liang
7. Jianwei Cui
8. Zuxuan Wu

Citations in the corpus

• Large Scale Image Completion via Co-Modulated Generative Adversarial Networks (2021, background): “…al., 2019a; Nazeri et al., 2019; Ren et al., 2019; Sagong et al., 2019; Wang et al., 2018; Xie et al., 2019; Xiong et al., 2019; Yan et al., 2018; Yang et al., 2017; 2019a; Yu et al., 2018; Yu et al., 2019; Zeng et al., 2019; Lahiri et al., 2020; Zhao et al., 2020a; Li et al., 2020; Zhou et al.,…”
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection (2021, methodology): “Perceptual Loss Perceptual loss is usually used in face inpainting approaches [37, 62] to measure the similarity between the restored”