Learning to Navigate the Energy Landscape

By Julien P. C. Valentin, Angela Dai, Matthias Nießner, Pushmeet Kohli, Philip H. S. Torr, Shahram Izadi, Cem Keskin. 2016

193 citations recorded by the source corpus.

In this paper, we present a novel, general, and efficient architecture for addressing computer vision problems that are approached from an 'Analysis by Synthesis' standpoint. Analysis by synthesis involves the minimization of reconstruction error, which is typically a non-convex function of the latent target variables. State-of-the-art methods adopt a hybrid scheme where discriminatively trained predictors like Random Forests or Convolutional Neural Networks are used to initialize local search algorithms. While these hybrid methods have been shown to produce promising results, they often get stuck in local optima. Our method goes beyond the conventional hybrid architecture by not only proposing multiple accurate initial solutions but by also defining a navigational structure over the solution space that can be used for extremely efficient gradient-free local search. We demonstrate the efficacy and generalizability of our approach on tasks as diverse as Hand Pose Estimation, RGB Camera Relocalization, and Image Retrieval.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Matterport3D: Learning from RGB-D Data in Indoor Environments (2017, methodology): “These previous RGB-D datasets have been used to train models for several standard scene understanding tasks, including semantic segmentation [27, 33, 13, 40, 7], 3D object detection [31, 24, 14, 36, 37], normal estimation [41, 23, 9, 3], camera relocalization [30, 39], and others [47, 11, 12].”
CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration (2021, unknown): “A.4 Data 3DMatch and 3DLoMatch 3DMatch [6] collects 62 scenes from SUN3D [45], 7-Scenes [46], RGB-D Scenes v.2 [47], Analysis-by-Synthesis [48], BundleFusion [49] and Halbel et al. [50], where 46 scenes are used for training, 8 scenes for validation and 8 scenes for testing.”
3DMatch: Learning the Matching of Local 3D Geometry in Range Scans (2016, unknown): “…keypoint volume pairs and their associated correspondence labels acquired from the data preparation described in Section 4, we minimize the cross-entropy error E = − 1 n n∑ i=1 [yilog(ŷi) + (1− yi)log(1− log(ŷi)] (1) over a training set of ground truth correspondences using stochastic gradient…”
InLoc: Indoor Visual Localization with Dense Matching and View Synthesis (2019, methodology): “This inherently leads to significant view changes between query and database images when comparedwith other existing datasets [76], [85], [90].”
PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors (2018, methodology): “This dataset is a large ensemble of the existing ones such as Analysisby-Synthesis [41], 7-Scenes [37], SUN3D [47], RGB-D Scenes v.”
Visual Camera Re-Localization From RGB and RGB-D Images Using DSAC (2020, methodology): “For feature-based competitors, we report results of the ORB baseline used in [3] and [81], as well as the SIFT baseline used in [81].”
3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions (2016, methodology): “3 of the main paper, we use registered depth frames of 67 different real-world scenes collected from Analysis-by-Synthesis [36], 7-Scenes [30], SUN3D [39], RGB-D Scenes v.2 [21], and Halber et al. [16], with 57 scenes for training and 10 scenes for testing.”
PREDATOR: Registration of 3D Point Clouds with Low Overlap (2020, methodology): “Dataset preprocessing 3DMatch : [56] is a collection of 62 scenes, combining earlier data from Analysis-by-Synthesis [44], 7Scenes [37], SUN3D [51], RGB-D Scenes v.2 [25] The dataset was captured using a Velo-dyne HDL-64 3D laser scanner by driving around the mid-size city of Karlsruhe, in rural…”
SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration (2020, background): “3DMatch is a RGBD-reconstruction dataset, which consists of 62 real-world indoor scenes collected from existing dataset [56, 49, 57, 34, 56, 11].”
Review: deep learning on 3D point clouds (2020, background): “This benchmark dataset is developed by Princeton University et al. It is a large collection of existing datasets, such as Analysisby-Synthesis [64], 7-Scenes [65], SUN3D [60], RGB-D Scenes v.2 [66] and Halber et al. [67].”
PPFNet: Global Context Aware Local Features for Robust 3D Point Matching (2018, methodology): “Real Datasets We concentrate on real sets rather than synthetic ones and therefore our evaluations are against the diverse 3DMatch RGBD benchmark [49], in which 62 different real-world scenes retrieved from the pool of datasets Analysis-by-Synthesis [43], 7-Scenes [39], SUN3D [47], RGB-D Scenes v.2 ”
3D-RCNN: Instance-Level 3D Object Reconstruction via Render-and-Compare (2018, methodology): “using separately-trained models for the bottom-up and inference stages [25, 60, 34, 51], we employ a single unified end-to-end trained network for inverse-graphics.”
The Perfect Match: 3D Point Cloud Matching With Smoothed Densities (2018, background): “Point clouds obtained from a pool of data sets [41, 32, 19, 39, 3] are split into 54 scenes for training and 8 scenes for testing.”
Learning Less is More - 6D Camera Localization via 3D Surface Regression (2017, methodology): “Training without a 3D model still achieves a good accuracy comparable to a sparse feature baseline (SIFT+PNP [27]).”