Multi-view Convolutional Neural Networks for 3D Shape Recognition

By Hang Su, Subhransu Maji, Evangelos Kalogerakis, Eric Learned-Miller. 2015

3,623 citations recorded by the source corpus.

A longstanding question in computer vision concerns the representation of 3D shapes for recognition: should 3D shapes be represented with descriptors operating on their native 3D formats, such as voxel grid or polygon mesh, or can they be effectively represented with view-based descriptors? We address this question in the context of learning to recognize 3D shapes from a collection of their rendered views on 2D images. We first present a standard CNN architecture trained to recognize the shapes' rendered views independently of each other, and show that a 3D shape can be recognized even from a single view at an accuracy far higher than using state-of-the-art 3D shape descriptors. Recognition rates further increase when multiple views of the shapes are provided. In addition, we present a novel CNN architecture that combines information from multiple views of a 3D shape into a single and compact shape descriptor offering even better recognition performance. The same architecture can be applied to accurately recognize human hand-drawn sketches of shapes. We conclude that a collection of 2D views can be highly informative for 3D shape recognition and is amenable to emerging CNN architectures and their derivatives.

Subfields: Computer Vision and Pattern Recognition, Geology, Industrial and Manufacturing Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Xiaogang Wang
  4. Ming-Hsuan Yang
  5. Jitendra Malik
  6. Trevor Darrell
  7. Andrea Vedaldi
  8. Andrew Zisserman

In Geology

  1. Leonidas J. Guibas
  2. Matthias Nießner
  3. Charles Ruizhongtai Qi
  4. Or Litany
  5. Armand Joulin
  6. Rohit Girdhar
  7. Ishan Misra
  8. Angela Dai

In Industrial and Manufacturing Engineering

  1. Leonidas J. Guibas
  2. Hao Su
  3. Charles Ruizhongtai Qi
  4. Michael M. Bronstein
  5. Yue Gao
  6. Yifan Feng
  7. Matthias Nießner
  8. Angela Dai

Citations in the corpus