Sparse 3D convolutional neural networks

By Benjamin Graham. 2015

225 citations recorded by the source corpus.

We have implemented a convolutional neural network designed for processing sparse three-dimensional input data. The world we live in is three dimensional so there are a large number of potential applications. In the quest for eciency, we experiment with CNNs on the 2D triangular-lattice and 3D tetrahedral-lattice.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Model Architectures, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Model Architectures

In 3D Vision & Pose

Citations in the corpus

OctNet: Learning Deep 3D Representations at High Resolutions (2016, methodology): “A similar approach is presented in [15, 16] where sparse convolutions are reduced to matrix operations.”
3D fully convolutional network for vehicle detection in point cloud (2016, methodology): “collected and clustered as the detection results. B. 3D FCN Detection Network for Point Cloud Although a variety of discretization embedding have been introduced for high-dimensional convolution [1], [8], for simplicity we discretize the point cloud on square grids. The discretized data can be repre”
Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks (2016, background): “An alternative approach that takes advantage of sparse representations can be found in [10] and [11], in which sparse convolutions are applied to comparatively small 2D and 3D crops respectively.”
Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs (2017, methodology): “Our approach is largely inspired by Graham’s sparse convolutional networks [15, 16], which enable efficient shape analysis by storing a sparse set of non-trivial features instead of dense feature maps.”
An End-to-End Transformer Model for 3D Object Detection (2021, background): “3D data can also be converted into a volumetric 3D grid by voxelization [1, 12, 15, 28, 35, 49, 56, 66].”
Deep Learning for Robust Normal Estimation in Unstructured Point Clouds (2016, methodology): “Techniques to efﬁciently apply CNN-based methods to sparse data have been proposed too [Gra15], but they mostly focus on efﬁciency issues, to exploit sparsity; applications are 3D object recognition, again with voxel-based granularity, and analysis of space-time objects.”
Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55 (2017, methodology): “We make use of sparse convolutional networks [4] to solve the task.”
Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation (2025, unknown): “The transition from early 3D Convolutional Neural Networks [30] to the now-common Transformer architectures [94, 67] has led to the development of classic shape generation networks, enhancing performance.”
O-CNN (2017, methodology): “It is possible to build similar hierarchical structures based on other lattices, such as the tetrahedral lattice [Graham 2015] and the permutohedral lattice [Jampani et al. 2016], the latter of which could be used for higher-dimension CNNs.”
Sparsity Invariant CNNs (2017, background): “Compared to their approach our sparse convolutional networks yield sig-niﬁcantly better results for depth upsampling while being as efﬁcient as regular CNNs. Graham [18, 19] and Riegler et al. [53] consider sparse 3D inputs.”
Submanifold Sparse Convolutional Networks (2017, background): “More recently, a number of convolutional network implementations have been presented that are tailored to work eﬃciently on sparse data [1, 2, 9].”
Learning Sparse High Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks (2015, background): “[23] propose extensions from 2D CNNs to 3D sparse signals.”
SECOND: Sparsely Embedded Convolutional Detection (2018, background): “In [25,26], a spatially sparse convolution is designed that increases the 3D convolution speed, whereas Ref.”
4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks (2019, methodology): “For this, we adopt sparse tensors [8, 9] and propose generalized sparse convolutions that encompass all discrete convolutions.”
3D Semantic Segmentation with Submanifold Sparse Convolutional Networks (2017, methodology): “Prior work uses a sparse version of the im2col operation that restricts computation and storage to “active” sites [4], or uses the voting algorithm from [22] to prune unnecessary multiplications by zeros [3].”
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision (2019, background): “Voxels can be easily processed by standard deep learning architectures, but even when operating on sparse data structures [23, 64, 74], they are limited to relatively small resolution.”
Recurrent Slice Networks for 3D Segmentation of Point Clouds (2018, methodology): “Benjamin Graham designed a sparse convolution network [5, 6] and applied it for 3D segmentation tasks [36].”
Occupancy Flow: 4D Reconstruction by Learning Particle Dynamics (2019, background): “While voxel representations can be easily incorporated in a deep learning pipeline, even variants which operate on sparse data structures are limited to relatively small resolutions of up to 256(3) or 512(3) voxels [23, 54, 61].”
Self-Supervised Pretraining of 3D Features on any Point-Cloud (2021, background): “However, since using convolution operations on point-clouds directly is difficult [31, 66], voxelized data is another popular input representation.”