Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images

By Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei David Liu, Yu-Gang Jiang. 2018

1,502 citations recorded by the source corpus.

We propose an end-to-end deep learning architecture that produces a 3D shape in triangular mesh from a single color image. Limited by the nature of deep neural network, previous methods usually represent a 3D shape in volume or point cloud, and it is non-trivial to convert them to the more ready-to-use mesh model. Unlike the existing methods, our network represents 3D mesh in a graph-based convolutional neural network and produces correct geometry by progressively deforming an ellipsoid, leveraging perceptual features extracted from the input image. We adopt a coarse-to-fine strategy to make the whole deformation procedure stable, and define various of mesh related losses to capture properties of different levels to guarantee visually appealing and physically accurate 3D geometry. Extensive experiments show that our method not only qualitatively produces mesh model with better details, but also achieves higher 3D shape estimation accuracy compared to the state-of-the-art.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

  1. Andreas Geiger
  2. Gerard Pons-Moll
  3. Michael Niemeyer
  4. Michael J. Black
  5. Jitendra Malik
  6. Matthias Nießner
  7. Angjoo Kanazawa
  8. Justin Johnson

In Computer Graphics and Computer-Aided Design

  1. Andreas Geiger
  2. Jonathan T. Barron
  3. Michael Niemeyer
  4. Matthias Nießner
  5. Justin Johnson
  6. Songyou Peng
  7. Matthew Tancik
  8. Angjoo Kanazawa

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Jitendra Malik
  4. Ming-Hsuan Yang
  5. Trevor Darrell
  6. Xiaogang Wang
  7. Andrea Vedaldi
  8. Andrew Zisserman

In Generative Models

  1. Andreas Geiger
  2. Jun-Yan Zhu
  3. Daniel Cohen-Or
  4. Eli Shechtman
  5. Richard Zhang
  6. Ming-Hsuan Yang
  7. Alexei A. Efros
  8. Jitendra Malik

In 3D Vision & Pose

  1. Andreas Geiger
  2. Michael Niemeyer
  3. Jitendra Malik
  4. Gerard Pons-Moll
  5. Justin Johnson
  6. Matthias Nießner
  7. Songyou Peng
  8. Michael J. Black

Citations in the corpus