OctNet: Learning Deep 3D Representations at High Resolutions
By Gernot Riegler, Ali O. Ulusoy, Andreas Geiger. 2016
1,618 citations recorded by the source corpus.
We present OctNet, a representation for deep learning with sparse 3D data. In contrast to existing models, our representation enables 3D convolutional networks which are both deep and high resolution. Towards this goal, we exploit the sparsity in the input data to hierarchically partition the space using a set of unbalanced octrees where each leaf node stores a pooled feature representation. This allows to focus memory allocation and computation to the relevant dense regions and enables deeper networks without compromising resolution. We demonstrate the utility of our OctNet representation by analyzing the impact of resolution on several 3D tasks including 3D object classification, orientation estimation and point cloud labeling.
Subfields: Aerospace Engineering, Computational Mechanics, Computer Vision and Pattern Recognition, Model Architectures, 3D Vision & Pose
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Andreas Geiger
- Vladlen Koltun
- Jitendra Malik
- Matthias Nießner
- Leonidas J. Guibas
- Marc Pollefeys
- Charles Ruizhongtai Qi
- Angela Dai
In Computational Mechanics
- Andreas Geiger
- Leonidas J. Guibas
- Matthias Nießner
- Michael Niemeyer
- Jitendra Malik
- Yaron Lipman
- Binh-Son Hua
- Sai-Kit Yeung
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Jitendra Malik
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Andrea Vedaldi
- Andrew Zisserman
In Model Architectures
- Andreas Geiger
- Chunhua Shen
- Xiaogang Wang
- Raquel Urtasun
- Jitendra Malik
- Yu Qiao
- Silvio Savarese
- Vladlen Koltun
In 3D Vision & Pose
- Andreas Geiger
- Michael Niemeyer
- Jitendra Malik
- Matthias Nießner
- Justin Johnson
- Angela Dai
- Jiajun Wu
- Leonidas J. Guibas
Citations in the corpus
- 3D Instance Segmentation of MVS Buildings (2021, methodology): “For example, OctNet uses an octree structure to avoid unnecessary cells [3], thus reducing memory consumption.”
- PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space (2017, background): “In 3D metric space, other than point set, there are several popular representations for deep learning, including volumetric grids [21; 22; 29], and geometric graphs [3; 15; 33].”
- Occupancy Networks: Learning 3D Reconstruction in Function Space (2018, background): “While it is possible to reduce the memory footprint by using data adaptive representations such as oc-trees [61, 67], this approach leads to complex implementations and existing data-adaptive algorithms are still limited to relatively small 256 3 voxel grids.”
- DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation (2019, methodology): “Octree-based methods [52, 43, 26] alleviate the compute and memory limitations of dense voxel methods, extending for example the ability to learn at up to 512(3) resolution [52], but even this resolution is far from producing shapes that are visually compelling.”
- Learning to Reconstruct Shapes from Unseen Classes (2018, background): “Apart from voxels, some researchers have also studied reconstructing objects in point clouds [Fan et al., 2017] or octave trees [Riegler et al., 2017, Tatarchenko et al., 2017].”
- Learning Shape Priors for Single-View 3D Completion and Reconstruction (2018, background): “While most of these approaches encode objects in voxels from vision, there have also been attempts to reconstruct objects in point clouds [11,15] or octave trees [40,49,39], or using tactile signals [53].”
- Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling (2018, background): “attempts to reconstruct objects in point clouds [12] or octave trees [44, 57].”
- OctNetFusion: Learning Depth Fusion from Data (2017, methodology): “Structure Module: As discussed above, the unpooling operation of the original OctNet architecture [36] has one major drawback: the octree structure must be known in advance to determine which voxels shall be split.”
- Hierarchical Surface Prediction for 3D Object Reconstruction (2017, background): “Similarly, a very recent work [28] which proposes to use octrees in a CNN, also assumes that the structure of the octree is given as input.”
- Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs (2017, background): “[28] data for training, and 20% for testing.”
- Neural 3D Mesh Renderer (2017, methodology): “are 3D extensions of pixels, are the most widely used format in machine learning because they can be processed by CNNs [2, 17, 20, 24, 30, 31, 34, 35, 36].”
- PointCNN: Convolution On X-Transformed Points (2018, background): “Octree [37, 47], Kd-Tree [22] and Hash [41] based approaches have been proposed to save computation by skipping convolution in empty space.”
- Deformable Shape Completion with Graph Convolutional Autoencoders (2017, neutral): “Additionally, voxel representations are often memory intensive and suffer from poor resolution [61], although recent models have been proposed to address these issues: implicit surface representation [12], sparse octree networks [57, 44], encoderdecoder CNN for patch-level geometry refinement [19], ”
- ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans (2017, methodology): “To more efficiently represent and process 3D volumes, hierarchical 3D CNNs have been proposed [30, 41].”
- A comprehensive survey on point cloud registration (2021, methodology): “OctNet [83] uses Octree to hierarchically divide the volumetric data into an unbalanced tree where each leaf node stores the feature presentation.”
- Sparsity Invariant CNNs (2017, background): “We further plan to investigate the effect of sparse irregular inputs for 3D CNNs [53]. such cases.”
- Point-Based Multi-View Stereo Network (2019, background): “Recent learning methods try to ease memory consumption by advanced space partitioning [21, 27, 24].”
- SO-Net: Self-Organizing Network for Point Cloud Analysis (2018, background): “Although the sparse design allows higher grid resolution, the induced complexity and limitations make it difficult to realize large scale or flexible deep networks [30].”
- MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation (2019, methodology): “OctNet [17] and O-CNN [19] utilize the sparsity of 3D data via the octree representations to save computation.”
- Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction (2017, background): “Riegler et al. [23] proposed to tackle the problem by using adaptive hierarchical octary trees on voxel grids to encourage encoding more informative parts of 3D shapes.”