SAL: Sign Agnostic Learning of Shapes From Raw Data

By Matan Atzmon, Yaron Lipman. Computer Vision and Pattern Recognition, 2019

598 citations recorded by the source corpus.

Recently, neural networks have been used as implicit representations for surface reconstruction, modelling, learning, and generation. So far, training neural networks to be implicit representations of surfaces required training data sampled from a ground-truth signed implicit functions such as signed distance or occupancy functions, which are notoriously hard to compute. In this paper we introduce Sign Agnostic Learning (SAL), a deep learning approach for learning implicit shape representations directly from raw, unsigned geometric data, such as point clouds and triangle soups. We have tested SAL on the challenging problem of surface reconstruction from an un-oriented point cloud, as well as end-to-end human shape space learning directly from raw scans dataset, and achieved state of the art reconstructions compared to current approaches. We believe SAL opens the door to many geometric deep learning applications with real-world data, alleviating the usual painstaking, often manual pre-process.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a163468fd1eca5ae1aa79046670a51efcc181954

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation (2021, background): “Neural implicit representations (NIR) have been widely adopted for learning shape and appearance of objects [4,10, 14, 20, 45, 50, 52, 56, 57], local parts [17, 18], and full 3D scenes [7, 11, 29, 55] from explicit 3D supervisions.”
3D Neural Field Generation Using Triplane Diffusion (2022, unknown): “They either solely learn geometry [1,4,5,10,11,15,21,23,45, 47, 48, 57, 65, 73] or use posed images to jointly optimize geometry and appearance [6,7,18,25,29,33,36–39,44,49,50,54, 55,59,67,72,82–84].”
Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance (2020, methodology): “…ray casting is mostly used with implicit shape representations such as implicit function deﬁned over a volumetric grid or implicit neural representation, where the implicit function can be the occupancy function [31, 5], signed distance function (SDF) [35] or any other signed implicit [2].”
Dynamic View Synthesis from Dynamic Monocular Video (2021, background): “, points in 3D space) to the desired quantities such as occupancy value [30, 45, 40], signed distance [38, 2, 31], volume density [32], color [36, 48, 45, 32].”
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction (2021, methodology): “We also studied the effect of Eikonal regularization [10] and geometric initialization [1].”
COTR: Correspondence Transformer for Matching Across Images (2021, methodology): “To achieve this, we leverage the inductive bias that densely connected networks possess in representing smooth functions [1, 4, 48] and use a transformer [73, 10, 14]”
Learning Gradient Fields for Shape Generation (2020, unknown): “Recently, SAL [5] learns a signed distance field using only point cloud as supervision.”
pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis (2020, background): “These can be supervised with 3D data, such as point clouds, and optimized as either signed distance functions [46, 38, 1, 16, 57, 20, 47, 55, 27] or occupancy networks [37, 6].”
Volume Rendering of Neural Implicit Surfaces (2021, methodology): “The geometry MLP gets a 3D position, x , and outputs the SDF value and an extra feature vector z of size 256, i.e., f ϕ ( x ) = ( d ( x ) , z ( x )) ∈ 1+256 . an We initialized f ϕ using the geometric initialization presented in [1], so that d produces approximated SDF to the unit sphere.”
AutoInt: Automatic Integration for Fast Neural Volume Rendering (2020, background): “Such representations have been explored for modeling shape parts [16, 17], objects [3, 6, 10, 19, 25, 27, 30, 34, 35, 40, 42, 43, 49, 53, 60], or scenes [14, 21, 29, 37, 45, 51].”
Space-time Neural Irradiance Fields for Free-Viewpoint Video (2020, methodology): “The core idea is to use a multilayer perceptron (MLPs) to implicitly model the occupancy [41, 42], signed distance functions [53, 2], object appearance [51], volumetric density [16, 36, 63, 67, 64, 44] in the 3D space.”
NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction (2022, background): “Concatenating the position serves as a geometry initialization [3] leading to a more stable learning of the geometry.”
Implicit Neural Representations with Periodic Activation Functions (2020, methodology): “Inspired by recent work on shape representation with differentiable signed distance functions (SDFs) [1, 4, 9], we fit SDFs directly on oriented point clouds using both ReLU-based implicit neural representations and SIRENs.”
I M Avatar: Implicit Morphable Head Avatars from Videos (2021, methodology): “We initialize the geometry network with geometric initialization [1] to produce a sphere at the origin.”
Unsupervised Learning of 3D Object Categories from Videos in the Wild (2021, background): “[44, 1] label each 3D point with its signed distance to the nearest surface point, [35, 6] predict binary per-point occupancies, [11, 10] proposed more structured occupancy functions, and [13, 60] reconstruct meshes from single views.”
Learning Continuous Image Representation with Local Implicit Image Function (2020, methodology): “This idea has been widely applied in modeling 3D object shapes [6, 28, 2, 13], 3D surfaces of the scene [42, 18, 36, 4] as well as the appearance of the 3D structure [33, 32, 29].”