PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization

By Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa. 2019

1,437 citations recorded by the source corpus.

We introduce Pixel-aligned Implicit Function (PIFu), an implicit representation that locally aligns pixels of 2D images with the global context of their corresponding 3D object. Using PIFu, we propose an end-to-end deep learning method for digitizing highly detailed clothed humans that can infer both 3D surface and texture from a single image, and optionally, multiple input images. Highly intricate shapes, such as hairstyles, clothing, as well as their variations and deformations can be digitized in a unified way. Compared to existing representations used for 3D deep learning, PIFu produces high-resolution surfaces including largely unseen regions such as the back of a person. In particular, it is memory efficient unlike the voxel representation, can handle arbitrary topology, and the resulting surface is spatially aligned with the input image. Furthermore, while previous techniques are designed to process either a single image or multiple views, PIFu extends naturally to arbitrary number of views. We demonstrate high-resolution and robust reconstructions on real world images from the DeepFashion dataset, which contains a variety of challenging clothing types. Our method achieves state-of-the-art performance on a public benchmark and outperforms the prior work for clothed human digitization from a single image.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

  1. Andreas Geiger
  2. Gerard Pons-Moll
  3. Michael Niemeyer
  4. Michael J. Black
  5. Christian Theobalt
  6. Jonathan T. Barron
  7. Songyou Peng
  8. Matthias Nießner

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Jitendra Malik
  3. Ming-Hsuan Yang
  4. Dacheng Tao
  5. Trevor Darrell
  6. Xiaogang Wang
  7. Andrea Vedaldi
  8. Andrew Zisserman

In 3D Vision & Pose

  1. Andreas Geiger
  2. Michael Niemeyer
  3. Gerard Pons-Moll
  4. Jonathan T. Barron
  5. Jitendra Malik
  6. Matthias Nießner
  7. Christian Theobalt
  8. Angjoo Kanazawa

Citations in the corpus